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FOREWOIU3 



This Uth annual report on Federally funded education programs administered by 
the U.S. Department of Education is transmitted to Congress in/esponse to 
several congressional mandates. Section * 17(a) of the General Education 
Provisions Act (GEPA) directs that the Annual Evaluation Report evaluate the 
effectiveness of programs in achieving their legislated purposes and include 
recommendations for achieying gredtecL- effectiveness; Section 12*6 of the 
Education, Amendments of 1978 (Pjblic Law 95-561) requires that reporting on 
program effectiveness also indicate compliance with provisions of the law on 
majfntenance of non-Federal expenditure^ where such provisions exist; and' 
Section 1305 of the Education Amendments of 1980 (Public Law. 96-37*) 
stipulates that the Afinual Evaluation Report ir.clude tabulations of available 
data to indicate the effectiveness of the programs and projects by sex, race and 
age of their beneficiaries. This report addresses these mandated requirements 
and, in addition, describes the means by which evaluation findings may improve 
education programs. 

A two-volume format has been adopted for the second consecutive year in order 
to respond to the specific needs of rej^ort users. Volume I, which is intended for 
general distribution, provides an overview of education evaluation activities in 
the Department of Education. This volume describes innovative information- 
gathering and evaluation techniques as well as management initiatives which will 
better utilize evaluation findings. There are overviews with generalizations 
from study findings and other analyses for programs in elementary and secondary 
education, postsecondary education, and special category programs. Highlights 
of new evaluation findings from specific studies are reported. There is also a 
special chapter on evaluation activities in the Office of Managment during fiscal 
year 1981. 

Volume II contains detailed program-by-program summaries of available 
information. For the first time. Volume II contains evaluation information on 
programs transferred to the Department from other Federal agencies in 1980 as 
a result of the Department of Education Organization Act. Programs 
transferred from the former U.S. Office of Education are again reviewed in 
Volume n, as they have been each year in past Reports. ^ ^ 

I would like to acknowledge the contribution of several offices and individuals to 
this year's report. Most program chapters in Volume II were prepared by staff of 
the Planning and Evaluation Service whose names appear at the end of program 
chapters as contact persons regarding program effectiveness. These staff 
persons worked junder the supervision of their Division Directors, Drs. Robert 
Maroney (Special Category, programs), Salvatore Corrallo, (Postsecondary 
programs), Janice Anderson (Elementary and Secondary programs) and Eugene 
Tu<5cer (Acting-Special Category programs). These Division Directors also 
contributed the Educatiorf Program Overviews in chapter II of Volume I. 
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Several staff members in the Office of Organizational Performance Service 
wrote materials for chapter IV of Volume I and for several program chapters of 
Volume n. The Evaluation Coordination Staff directed by Mr. Edward Classman 
and including Ms. Elaine Creen, Ms. 3ean Onufry, Ms. Yvonne Briscoe, Ms. Diana 
Carpenter, and Ms. Pamela Butler, drew together and organized the report's 
content from a varie'ty of sources while preparing, editing, and typing the 
materials of Volume I. A final word of recognition goes to the secretaries in 
Planning and Evaluation Service and in Organizational Performance Service who 
typed the many revisions for Volume 11. 



Gary Jones 

Deputy Under Secretary for 
Planning, Budget, and Evaluation 
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Secretary's Summary ^ 

The Annual Evaluation Report of the Department of Education is prepared each year 
in response to a statutory mandate of ^ the Congress. The law calls for evaluation of: 

**• • • the effectiveness of applicable programs ... in achieving their legislated 
purposes together with recommendations >yhich will result in greater effective- 
ness in achieving such purposes." 

This mandate Ts~Hi5Uenging, not on ly^ because^ it is broad but also because evalur.tors 
have often had to invent new ways of describing and measuring education effective- 
ness in quantitative terms. The Department evaluation studies surrimarized in this 
eleventh annual Report represent a remarkable advance in reliability? and validity of 
knowledge about Federal education programs during the last decade, in terms of 
describing how programs work and what ef^fects they create. For example, this Report 
provides information which was not available a decade ago about some characteristics 
oi successful compensatory programs. Yet Ave still have much to learn about what- 
specific steps a school district could take ^Jo improve effectiveness. 

The most significant message is that much has been learned from evaluation studies. 
Many readers of the Annual Evaluation Report refer to its two volumes to find the 
material on a single program. They miss the larger picture. This Summary has been 
prepared as a comprehensive and convenient reference of information about program 
effectiveness across a wide array of evaluation studies in compensatory education 
tincluding urban areas and bilingual programs), school desegregation, postsecondary 
student and institutional aid, and adult education programs. 

The information contained in the Annual Evaluation Report to the Congress is based on 
studies that have been completed most recently or are still undprway. In some cases, 
the most recent studies concluded several years ago. Nonetheless, these reports are 
still valid and, in most cases, refer to programs or situations that have been relatively 
stable over time. The advantage of including information from -older studies, when 
later ones are not available, is that each edition of the Report is a compendium of 
informatioii available for each program administered by the Department of Education. 

The following sections of this Summary present highlights of this year's Report for 
several of the Department's major programs. References in parenthesis are to 
chapters of Volume I or Volume II which provide additional information. 

Title I of the Elementary and Secondary Education Act 

This program provides financial assistance to local educational agencies serving areas 
with concentrations of children from poor families, in order to meet "the special 
educational needs of educationally deprived children." 

♦ General Findings ^ 

Evidence about the effectiveness of Title I services is now available from several 
sources at the Federal, State and local levels. These sources show that, in general. 
Title I Services are well-targeted in terms of schools in poor areas and of low- 
achieving youngsters, are supplemental to services provided by States and school 
districts, and are effective in helping many^ participating students to progress beyond 
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what would be expected without the program. (Volume I, Chapters 11 and III; Volume 
Hi Chapters Al, A2, and A3) 

o Specifically, the program is found most often in schools in poor 
neighborhoods: for example, 84% of the nation's elementary schools with 
more than half of their students from poor families offer Title I programs; 

o Title I programs concentrate their services on low-achieving students. 
Results in 1977 from a national study showed that nearly 53% of pupils 
served in grades one through six are in the lowest quartile in reading and 
math achievement and that 8*% of pupils are in the lower half nationally. 
Results from all States for the 1980-81 school year showed that average 
pretest scores for students in Title I projects in reading and m^th, grades 
two through six, ranged from the 22nd to the 27th percentile; 

o According to'' recent repot^jts from the States, approximately 5A million 
students from pre-school through grade twelve received Title I services 
during the 1980-81 school year. The reports also show that about 4.2 
million or 78% of those children received services in reading while 46% 
received services in math. The national study further showed that 
resources allocated to Title I programs in reading and math were more, than 
times as great as those allocated to the regular programs in those 
schools; 

o The national study, the State reports, and data from the National Assess- 
ment of Educational Progress, all show that Title I services are often 
effective in improving student performance in reading and math. The 
national study found Title I students gaining in reading by 10% to 17% more 
than similar non-Title I students in grades one through three. In grades 
four through six, however, the gains of Title I students relative to those of 
similar non-Title I students ranged only from 0% to 8% greater. In math, 
the relatively greater growth of ^Title I students ranged for grades one 
through six f ro|n 9% to 74%; 

o State reports based on school district data indicate greater gains by Title I 
students, relative to similar students in national norm groups, ranging from 
3% to 44% in reading and math and involving grades two through eight. 
These results are based on a 12-month interval from pre-test to post -test. 
Results based on, a 9-month testing interval for the same grades also 
showed relatively greater gains for Title I ranging from 67% to 198%, but 
possible problems in methodology are grounds for caution in interpreting 
these findings; 

o . The National Assessment of Educational Progress has evidence of improve- 
ments during the past four years in the educational status of nine-year-olds 
from minority groups, and of improved achievement levels in Title I 
schools. These gains in achievement may be attributable, at least in part, 
to increased attention to basic skills and to /effective programs in 
compensatory education. / 

♦ Recent Results from Urban Areas 
(Volume I, Chapter 11; Volume 11, Chapter Al) 

o Evidence from local sites often confirms the national findings. For a 
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sample of^ 16 Community School Districts in New '^'ork City, student gains 
increased by 30% in comprehension and by 16% in vocabulary between 19S0 
and 1981, under a new competence testing program that stressed reading 
comprehension. During the same period, city wide test scores, as well as 
"results for N^w York State on the Scholastic Aptitude Test, showed 
sufficient improvement to surpass national averages for the first time in 
years; - 

o During each oi the past two school years, the typical Title I student in 
Chicago improved in reading by at least four percentile points as measured 
by standardized tests. Students in some schools did even better, with gains 
up to 67% greater than those of similar students in appropriate norm- 
groups. In addition, 80% pf the parents of these Title I children agreed 

^ with the teachers and principals at the schools in endorsing the Title I 
projects; S 

o In New Jersey, Basic Skills Improvement (BSD programs represent 
compensatory education programs funded through combinations of ESEA 
Title I^ State compensatory education, and school district efforts. New 
Jersey reported that not only had the programs accelerated the acquisition 
of basic skills, but that the acceleration was greater in 1980 than it had 
been since the State first began evaluating its programs. In reading, BSI 
\ students improved their performance from an average pretest ;nean at the 

\ 23rd percentile to a posttest meLy at the 36th percentile. In mathematics, 

students moved from the 26th peicentile at the beginning of the program 
to the *2nd percentile in the following spring; ^ 

o Effective classrooms tend to be those with teachers actively engaged in 
instruction, with disruptions kept to a minimum, with frequent feedback on 
student progress, and with teacher participation in curriculum planning and 
evaluation. 

* Recent Results from State-administered Programs under Title I 
(Volume n. Chapters A2 and A3) 

o The State-administered program for neglected or delinquent children 
involved approximately 630 institutions in 1980 and is also well-targeted on 
low-achieving students. It has been effective in improving the attitudes of 
youngsters in institutions with regard to school and to themselves as 
learners. In sites that emphasized careful scheduling of instructional 
activities and where teachers had relatively less non-instructional 
responsibility (e.g., accompanying students around the institution) 
made appropriate use of audio-visual materials for instruction, studi^nt 
progress was better than the national average for this group. Nonetheless, 
from a national perspejctive, problems in Jthe delivery^of services to these 
youngsters appear to be limiting student progress under this program. In 
response, the Department of Education is attempting to make available 
models of effective service delivery and to improve project monitoring at 
these sites; 

o The State-administered program for migrant children has also been serving 
greatly disadvantaged children. A national study al^o found, however, that 
approximately half of the youngsters seryed attended the same school 
throughout the school year; of those who did not attend the same school, a 
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little more than half (i.e., about 30% of all children served by the program) 
missed only an average of six weeks from their main schools. In response, 
, program administrators are examining their recruitment methods in order 
to locate and then serve the more mobile children among the population 
eligible for services; 

o Students in the migrant education program tend to be older than their 
classmates and their test scores on average for grades two, four^ and six 
range from the 21st to the 25th percentile; 

* o The home language for these students is often not English, but teacher 
ratings (confirmed by some testing) indicated that language problems 
interfered with schooling for only 26% of those children. 

Title VI of i the Elementary and Secondary Education Act (Emergency School Aid Act) 

This program, recently included in the block grant program under the Education 
Consolidation and Improvement Act of 1981, has funded efforts to meet special neecis 
related to school desegregation and to encourage voluntary elimination of minority- 
group isolation in schools. (Volume II, Chapters AiO and Ail) / 

o Recent studies indicate that 329 school districts, or apfiroximately 65% of 
those that applied, received Basic Grants under this pr6gram in fiscal year 
1980,_^Authorized activities included hiring and training of staff, deyelop- 
ment of instructional materials, and activities in community relations; 

o Students who participated in human-relations activities funded under this 
program Showed greater improvement than their peers in inter-group 
attitudes and behaviors as well as in self -concept; 

o Three of fifteen school districts visited in a study of discipline components 
under this program indicated reductions in target schools of 
disproportionate disciplinary actions such as suspension, expulsion, and 
corporal punishment. The three sites appeared to have str<?ng central 
administrations and active particiipation by parents and staff in program 
planning; 

o For fiscal year 1983, all school districts that receive grants under Chapter 
2 of the Education Consolidation and Improvement Act will have the 
option, depending on local needs aM priorities, to fund activities similar to 
those of the former Title VI program. 

Title VII of the Elementary and Secondary Education Act 

This program has the general goal of developing proficiency in English for children of 
limited English-speaking ability and providing them with equal educational 
opportunity. (Volume I, Chapter III; Volume 11, Chapter Bl) 

o A national sttidy completed in 1977 found that fewer than one-third of the 
, students then served were actually of limited proficiency. In response, the 
Congress and program administrators took steps to improve the targeting 
of program services; 

o No new nationally representative data about the effectiveness of the Title 
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VII program have been collected since the Education Amendments of 1978 
clarified* eligibility for services. Nonetheless, ^ a recent reanalysis of 
existing information about a variety of instructional strategies for students 
^ ol limited-English-proficiency has emphasized the importance of allowing 
and encouraging decisions by State and local educators regarding student 
needs and instructional strategies to meet those needs; 

o The program currently serves approximately 223,000 students who 
collectively represent 71 different languages; 

o The Department plans to begin this year a mandated multi-year study of 
student progress. In the meantime, information on Title VII projects is 
being compiled and analyzed so that useful data will be available before ^ 
completion of the multi-year evaluation. 

Student Aid Programs, in Pojtsecondary Education 

The primary gdal of the Departm^nt^of Education's ppstsecondary education programs 
in fiscal year 1981 was to increase eaucational opportunity. The strategy for 
achieving this goal involved the awarding of financial assistance to students. (Volume 
I, Chapters IJ and III; Volume II, Chapters El through E6) 

o In recent years, the/ rates of growth of enrollments in postsecondary 
education have steadry decreased. In 1980, in effect the first year of 
operation of the Middle Income Student Assistance Act, the downward 
trend in these rates was reversed for both men and women; 

o In academic year 1979-80, the student burden (defined as the cost of 
attending a postsecondary institution less all nnnrieturnable aid such as 
grants and family contributions) for most dependent students was less than 
the amount that a student could reasonably be expected to earn or borrow. 
This finding suggests that financial barriers to postsecondary education 
have been overcome; 

o Although student burden w^as generally larger for independent students as 
opposed to dependent students, most students appear to have had only 
limited financial difficulty in attending all but the most expensive private 
— ins^titutions; 

(J Federal stpdent aid programs appear to provide sufficient funds to equalize 
the level of the student burden at institutions with similar costs for 
families with annual incomes of less than $2*,000; 

^, 

o . Federal aid programs appear to have placed attendance at relatively high- 
cost institutions within the financial reach of the poorest students, 
although it still entails considerable sacrifice, especially for independent 
students; 

> 

o A redesign of the current delivery system for student aid programs at the 
Federal level, in coordination with stated, educational institutions, and 
lending institutions, would result in a morelnearly equitable distribution of 
funds and in reduced opportunity for fraud and abuse; 

o Studies of Federal programs designed to remove non-financial barriers to 
college attendance continue to show that these programs are an effective 
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way^fdfincrease the enrollment and persistence of participating students in 
:secbndary institutions. 

Institutional Aid Programs in Postseconciary Education 
\ 

Although most Federal aid to postsecondary education goes directly to students, a . 
small portion has been directed to improving the quality of education offered to 
students* 

o Projects of the Fund for the Improvement of Postsecondary Education 
(FIPSE) at educational institutions were typically adopted in whole or part 

\ by -six other institutions* However, 80% of the FIPSE project directors felt 
the project would either not have begun or would not have survived without \ 
the Federal grantt (Volume II, Chapter E25). 

o The Department of Education has dieveloped a procedure for self -assess- 
ment by postseccindary institutions which has effectively reduced the heed 
for Federal assistance in strengthening State licensing and oversight 
'activitiefs. At present. State licensing agiencies, private accrediting 
associations, and post secondary education institutions are voluntarily using 
that procedure as ^ self -test in such areas as tuition rebate procedures in 
cases of student withdrawal, catalogue descriptions of admissions 
/procedures and ra^es, claims regarding job pl^cen^ents, and practicefs in 
student financial assistance (Volume II, Chapter) E31). 



o A study of the fiftapcial condition of instjKitions of higher education in 
fiscal year 1979 incMcated that most sch(5ols wpre not then in immediate 
/ financial difficulty. I^ there were a substantial jreduction in the funding of 
Federal student-aiid programs, however, enrollments could be affected with 
\ prd^ound effects on the financial condition of a substantial number of 

\ institutions (Volume I, Chapter III). j 

Adult Education - Grants to States 

This progra^^ has the goal of expanding educational^ opportunities for adults who need 
to acquire basic skills, who want to continue their'^ducatibn at least tp completion of 
secondary school, or who want training to become "more employable, productive, and 
responsible citizens." (Volume I, Chapter III; Volume II, Chapter D6) 

■ • * 

o During fiscal year 1979, approximately 1.9 million adults receiveid services 
from the Adult Education program at an average cost to. the Federal 
Government of $*6 per participant. Approximately *1% of these 
participants indicated they had fully attained their personal goals in 
entering the program, while \ an additional 38% of these participants 
indicated they had partially attained their goals; 
* \ • 

o Much of the instruction in the program, especially when "English as a 
Second Language" is not inyolvedi is individualized in nature and involves a 
competency-based approach to adylt education; 

\ - ' \ ' 

o. Although schopl jgfistricts administer the great majority of adult education 
' u ' projects, only 38% of students meet in .classrooms of elementary or 

; secondary schools, while i?% me^t in aduit learning centers, 10% in 
community colleges and vocational/technical schools,, and 25% in churches, 

ERJC ^ , \ ij. 
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prisons, libraries, or private ^homes.^ This finding contrasts with the 
widespread belief that most participants were mleeting in classrooms of 
elementary or secondary schools; , f 

Thtere appears to be littleor no support I for requiring special certification 
for teachers of adult education;^ / 

Although sortie participants pointed to a need for incneased child-care and 
transportation services, most respondents clearly do not share those needs; 

In some States there is a good Federal-State relationship in operating l:he 
program, while in other States the relationship is spotty, - Relationships 
between States and School districts were generally quite good, with 
communications judged asrhelpTuL There were virtually no relationships 
between the Federal and local levels of th^prpgram, / 
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CHAPTER I 

\ 

Evaluation in the Oepartmient of Education 
During Fiscal Year 1?8I 



Evaluajtion is a critical^ component of Federal education activities. The_ 
impor'^ance of evaluation in the Department was recognized by the Department 
of Education Organization Act, which states as one of its purposes "to promote 
improvements in the quality and usefulness of education through federally 
supported research, evaluation, and sharing of information." 

Broadly defined, evaluations are objective assessments of program and manage- 
ment performance. Specific approaches adopted, costs incurred, and study 
objectives vary. But all ED evaluation activities share the common purpose of ^ , 
trying to ensure that Federal education monies areieing efficiently spent on y 
programs that me^t their mandates. ^ 

The Depairtrhent has undertaken a series of initiatives that bring together 
evaluation/ policy, and budget staffs to explore the implications of program 
evaluations on all aspects of current departmental activities and to incorporate 
evaluation results into future planning, ifi number of management evaluation 
efforts, also highlighted in this rep()rt, gather information on program operations 
that can be used in conjunction with the more elaborate program evaluation 
studies. . . - ^ 



Changes in Organization and Mission During Fiscal Year 1981 



Midway through Fiscal Year 1981, the Office of Evaluation and Program 
Management (OEPM) in the Office of management was reorganized. The three 
divisions responsible for program evaluations (the Elementary and Secondary 
Programs Division, the Post Secondary Programs Division, and the Occupational, 
Handicapped, and Developmental ProgrSms Division), together with the 
Evaluation Coordination Staff, tt;ansf erred f com the Office of Management to 
what had been the Office of Planning and Budget. That newly augmented office 
changed its name to the Office of Planning, Budget, and Evaluation (OPBE). The 
evaluation units joined with the Division of Analytical Systems and the Division 
. Qf Technical Systems already in OPBE to form a new Planning and Evaluation 
Service (PES) which provides planning, analytic, evaluation, and policy analysis 
capability ahd services to the Department. The Director of the new OPBE is the 
Deputy-UnderSecretary for Planning, Budget, and Evaluation. 

The remaining divisions of OEPM, which had made up the former Office of 
Management Evaluation, became the core of a new Office of Organizational 
Performance Services (OPS) in /the Of f ice of Management. Made up at first of 
-^e Division of Quality Assuranj^e, Division of Program Assessment, Division of 
Management Analysis, Division of Organizational Development, and Division of 
Education Data Control, OPS was itself reorganized late in the fiscal year to 
^.^clude three main divisions: 1) the Division of Education Data Control, 2) the 
Division of Organizational Development and Analysis, and 3) the Division of 
Performance Management Systems. The Office of Organizational Performance 
Services reports to the Deputy Undersecretary for Management. 

An important characteristic of this year's Annual Evaluation Report was the 
close collaboration of the management evaluation divisions with the program 
revaluation divisions in producing materials for the Report. This collaboration 
included not only the overviews on management evaluation activities that are 



^ptes^nted in yqlume I, but also a number of sections on individual Department 
programs in Volume II. The mutual assistance of the program and management 
evaluation units was an indispensable feature in the timely production of the 
fiscal year 1981 report. \ 

r ■ _ . • 

. The ED Poliicy Analysis Agenda for Fiscal l ears i98i--i983 

e> ... 

Fiscal year 1981 \yas characterized not only by the aforementioned internal 
organizational changes, but also by a national election and the transition to a 
new Administration. The Administration is intensively reviewing and extensively 
altering the concept of the appropriate Federal role in education and the 
appropriate rank and strud^ure of the Department itself. New legislation has 
focused on funding levels for existing programs, block grants to State education 
agencies and to school districts^ program consolidation, and reduced constraints 
on the use of Federal funds by State and local agencies. 

The Department proceeded with defining its future information needs, 
identifying information already available from past studies and analyses or sooh- 
fb-be available from current activities, and selecting appropriate projects^for 
fiscal years 1982 and 1983, " The organizational changes referred^ to^rfiade it 
possible to plan for evaluation, and to plan for^poiicy analysis^^ a broad 
context of^the budget and analytic functions of the Department. The resulting 
Policy Analysis Agenda is-meant to address., major policy and budget questions 
and to ensure that policy and budget decisions will benefit frfom the results of 
evdluations, planning studies, technical analyses, anci policy reviews. 

The Policy Analysis Agenda developed during fiscal year 1981 for future fiscal 

years has the following objectives: . 

_ . i - 

o to identify the important issues that face Department of Education 
(ED) programs and existing policies: 

o to determine how these issues should best be analyzed; 

o to establish priorities among the information needed to make 
program cind policy decisions, to plan Department activities, and to 
propose budgets, legislation, and other Government, action; 
consequently, 

o to determine the level and allocation of resources among items in the 
analytic agenda. 

Preparation of the Policy Analysis Agenda thus had four distinct but related 
aspects during 1981. One involved working closely with ED program and staff 
offices to identify major issues in programs and management that the 
Department would have to address during at least the next 2 fiscal years. 

The second involved the documentation of planning, evaluation, and analytic 
projects conducted during fiscal years 1980 and 1981. The objectives of the 
second aspect were twofold: to describe available or sbon-tO'-*be available 
information to help deal with the issues already identified; and to prepare 
responses to the annual survey conducted by the Office of Management and 
Budget (OMB) on resources used by each Executive Agency for evaluation and 
management improvement,' and to the annual survey by the General Accounting 




Off ice (G AO) of Federal evaluation projects. 

The third aspect involved the proposals by evaluation, program, planning, and 
other staff offices of studies, analyses and other projects designed to provide 
still-nwded information in coming years tp help address the Agenda issues. Of 
considerable assistance in this planning phase were recommendations regarding 
evaluation issues and projects made by the Committee on Evaluation and 
Information--S5rstems.<CEIS) of the Council of Chief State School Officers, In 
addition, program and staff offices in ED provided information on organizational 
resources, structures and activities thai they intended to devote to evaluation or 
planning studies during future fiscal years. 

The final aspect during 1981 involved the preparation by OPBE of a draft Policy 
Anallysis Agenda for Fiscal Years 1982-83, review and comments by program and 
staff top management, decisions as needed by the Secretary and Under Secretary 
of ED about the content of the Agenda, and preparation by OPBE of the final 
Policy Analysis Agenda for Fiscal Years 1982-83. 

The schedule for development of the Agenda was designed to complement the 
calendar for fiscal year 1983 budget formulation. Much of the information 
generated by each process was useful to the other. Tlie period of preparation of 
the fiscal year 1983 budget for the Department was an appropriate time to 
consider program and policy questions which needed further analysis^fpr 
resolution. 



I 



CHAPTER n 



EDUCATION PROGRAM OVERVIEWS 



1 



ERIC 



17 



Elementary and Secondary Education Programs 

Evidence on Program Components Hypothesized 
To Be Related to the Quality of Education 

Fiscal year 1981 the completion of several evaluation efforts and the re- 
analysis or further interpretation of results of earlier evaluations. The growing 
information base and pressing budgetary debates both forced and facilitated a 
switch in orientation, in many cases from, "Does this program work?" to "How do 
the strategies iii this program work?" 

Attention is focused, therefore, to discussion of what we^know about a variety of 
program components, most notably those /hypothesized to lead to the 
accomplishment of program goals — and therefore required by law. One sees in 
the Elementary and Secondary Education Act (ESEA), for example, a global 
model of a program in which various components are required — plans, 
coordination of services, training, instruction (or another form of servtce), 
evaluation, dissemination, and parent involvement. Because several studies have 
examined program outcomes and the degree to which various components seemed 
to contribute favorably to those outcomes, and because a few studi^es have 
a focused on specific components such as evaluation or parental involvement, we 
^ now have pieces of evidence about many of the separate components. 

The^. full set of components are depicted in Exhibit 1 with legislative provisions 
requiring them, for the programs authorized in the Elementary and Secondary 
Education Act. A clear picture emerges from Exhibit 1 of strategies being 
common requirements. Of interest is the <legree to which .they seem to lead to 
program success or the improvement of educational quality. 

Following are the major, findings about the relationships among various program 
components and the accomplishment of program goals — most notably in the 
area of student achievement, the amelioration of intergroup relations and the 
improvements of attitudes during school desegregation efforts. Research is also 
underway in bilingual educ£ition to identify components related to program 
quality. j 

This overview is very brief and the findings are gleaned from several studies — 
any one of which typically focuses only on a subset of the relationships among 
this larger set. Therefore, the reader is cautioned not to jump to' any quick or 
simple conclusions. The relationships have been documented, but in most^ases 
only one at a time; that is, one cannot say from this evidence that all these links 
hold for all combinations of the components in all settings. 

In the fifteen years since passage of the Elementary and Secondary Education 
Act, several studies have documented the relationship between program 
participation and increased performance in the basic skills. Most recently, for 
example, investigators conducting the Study of the Sustaining Effects of 
Compensatory Education reported that Title I students in grades 1-3 showed 
greater progress in reading than would have been expected for them without the 
extra help offered by Title I; such progress over and above regular growth was 
also documented in math for Title I students' in grade 1-6. Factors responsible 
for this extra progress were found to be exposure to teachers with more 
experience, more time in regular "non-compensatory" instruction, and work with 



EihlbU 1* Components \J Convnoiily Required In Prog'^ams sponsored under 
the EleMent4ry ind Secondary Educathtn Act 



Education of the Disadvantaged 
Basic Skill Improvenent 
Hetric Education 
Preschool Partnership. 
Consumer Education 
Touth Employnent 
Law«related Education 
Envlronriental Education 
Correction Education 
Biomedical Sciences 

Population Education 
Ifnprovement In Local Practices 
O Guidance. Counseling. Testing 
Emergency School Aid 
Bilingual Education Programs 
Conniunlty Schools ^ 
Gifted and Talentec^ 
Education Proficiency Standards 
Uonen's Educational Equity 
Special Grants for Safe Schools 
Ethnic Heritage 



Law i 
Title I 

Title ) i (Parts A. B) 
Ultle III (Part B) 
Title III (Part 0) 1/ 
Title 111 (Part E) 
Title HI (Part F) 
Title III (Part G) 
Title III (Part H) 
Title III (Part J) 
Title III (Part L) 
Title III (Part N) 
Title IV (Part C) 
Title IV (Part 0) 
Title VI 
Title VII 
Title VIM 
Title IX (Part A) 
Title IX (Part B)' 
TIt/e IX (Part C) 
Title IX 



Coordlnr 



t/, 



t 0> 



. tie IX (Par' E) 



808(a) 

921(a.2) 

944(a.l) 



124(f) 



341 (b. 3). 



808(a.2) 



953(1,0 



Training 
124(1) 



333(b,l,0) 
341 (b,2) 
347(d,4) 

351(c), 353(b,2,0) 
3/2(a) 

392(b,l) 

43l(a.7), 43l(b,C) 
4^l(i.2,8) 
6a7(a,l) 
«l(i.3,A)5/ 
See footnote 5 



922(a,2) 

932(a,l,B), 932(a.l,E) 
944(a,l) 



Instruction 
I24(a,f,2) 
205, 221(2) 

325(b.l) 

>33(b,l,C) 

34l(b,2), 



Evaluation 




607(a, 2-4,8) 
72l(a.t) 

/ 



y Entries In the table are section nunters of the law / 
2/ An Individual Ized educational plan for every participant is encouraged* 

3/ Since the Title III programs require applications for funding which describe the purposes, strategies and 
" and evaluation to be supported under the grant, thny could be said to require a plan* However, there Is no 
foraial requireinent in the law for a plan which is reviewed by local interested persons (as In ECIA, for 
example) or individualixed educational plan for each student (as in Public Law 94-142, the Education for 
all Handicapped Act, or in Section I2g of ESEA Title I), so this column has been left blank for the 
Title III programs. 

\ y) y Not all parts of all titles are included because for ^soine, such asVitle 11. Part K for Oissemination or 
Title V'for State Leadership, the programs are not elementary and secondary education per se. or due to 
their nature, th^se program components make no sense* \ 

5/ Bilingual education also has an entire program under Section 723 devoted to training educational personnel 
In Institutions for higher education to prepart them for service in tne area of bilingual tdUcation* This 
In addition to the In-service training allowed under local grants. AUiallar training prograa is autliorited 
Q under Section dl2 to train cpanunity education personnel. ^ \ 



Dissemination 
124(h) 



Parent Involvement 
I24(j,l25) 



124(g) 

202(a,2), 221(4) 202(a,3) 209, 222(?) 206.221(3) 

313(3) 3|2(a.3) 

325(b.3) 325)c) 

333(b.2). 334(3). 333(b.l.B) 
335 

347(d.2) 

34l(d.:5) 

351(c). 354 (a.3) 351(c). 353(b.2.B) 
372(a) 



3B4(a.n) 

3g2(b.3 and 5) 

44l(a.2.G) 

607(a.6). 610^ 

(a.ll) 
721(b.3.C.iii) 

B08(a.ll) 

gd5(a.6) 

g22(a.3) 

932(a.l.c) 

944(a.4) ^ 

4 



385(b) 
392(b.2) 



607(a.6) 




384(a.8B) 

43l(a.4). 43l(b.lB) 

6ID(a.l) 
703(a.4.E) 
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tutors. The relationships were ,weak enough, however, to highlight the 
importance of indepth, observational efforts to identify truly effective 
instructional practices. ♦ 

The body of knowledge abi^ut the importance of specific teacher practices and 
environmental factors for promoting student growth is large. Wiley and 
Hamischfeger found that/' student gains were related to time spent in instruction 
(1974) as did Hanson ana Ross (1975); more recently, measures of "educational 
time" have been refined, however, to be 'time-on-task" rather than "allocated 
time", and as would hp expectdU, stronger relationships between this measure 
and student gains have been found (Alvin and Roth, 1978; Marliave, Fisher, and 
Dishaw, 1977). l^esling and Pfannenstiel vertified this relationship in 
classrooms in institutions for the neglected or delinquent and found that the 
critical factor in ^hat setting for increasing student time-on-task was having 
teachers actively teach. ^ That is, the use of audiovisual materials (oft n misused 
to allow the teacher's absence from the classroom rather than as a vital part of a 
lesson plan) ana of independent seat work were found to negatively affect 
student growth (1981). 

■ ' / - ■ 

Similarly, an/indepth study in second and fifth-grade classrooms in 55 poverty 
schools documented positive and negative classroom influences: teachers who 
spend a^^reater percentage of their time in instructional activities (as opposed to 
non-instruptional ones such as behavior management) are more effective in 
keeping their students attentive to tasks and such attentiveness is related to 
greater Achievement growth; students assigned independent seatwork are more 
frequently "off task"; and in groups with great amount of disruptions, the sources 
of thox disruptions could more frequently be under the teacher's control (Lee, et 
al., 1^81). ^ ~ 

This body of evidence about the importartce of teacher's being active instructors 
and managing classrooms to reduce d|i5ruptions suggests that use of better 
classroom management strategies and teacher -training activities to increase the 
^repertoires of skills available to teachers are two ways to improve programs. 
Driven by this and other research, the National Institute of Education is 
sponsoring a demonstration program in which local educators refine and use 
particular models of classrooifi management techniques (Kocher, 1981). Data 
from this experience, as well as complete descriptions of the models themselves, 
will be disseminated for use by other local educators- deciding to try the same or 
similar routes to program improvement. 

The role of teacher training as a component in successful programs was 
documented in the Study of Emergency School Aid Act Human Relations 
Activities. Specifically, staff training and compensation for training were 
related to positive outcomes in students' attitudes toward school, their 
intergroup behaviors, and their intergroup attitudes (Doherty, 1981). 



A detailed report on the findings of this study regarding the effectiveness 
of Title I services is given on pages 40 to 45 of this volume. 
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Occasionally a plan for coordination of services is required by law. Seen as a 
way to demonstrate that coordination - has occurred or does occur in the 
educational programs f>r certain groups of students, a plan may be required' as 
documentation of the goals and strategies to be implemented across an entire 
school or district^ or more as a type of prescription for services to be provided 
for each student based on an individual assessment of needs. 

Rubin and David (1981) examined schoolwide projects implemented under Section 
! 133 of Title I of^'ESEAy Which requires a plan including an assessment of student 
needs, a comprehensive program to meet those needs, input from all staff and 
parents, consultation throughout the project among all parties, appproval by the 
school advisory council, and evaluation procedures (ESEA Title I, Section 133 
(b)). They found that planning to meet student needs by a program of 
comprehensive services from various resources did not occur more in those sites 
implementing schoolwide projects than it had before the site's decision to take / 
advantage of the new provisions. In places where systematic planning did occur^ / 
it was not due to the requirement but rather to the administratorfs seeing a plan / 
as necessary for the effective use of resources and for consensus-building among / 
staff and parents. 

Evidence to support the usefulness of coordination of services as outlined, by a/ 
plsm was found in the in-depth substudy of the Sustaining Effects Study noted 
above. Specifically, the more regular and compensatory teachers with secondi 
grade students in common showed knowledge (on a questionnaire) of the 
curriculum being addressed by the other, the greater were student gains in botli 
reading and math (Lee, et al., 19S1). The results did not hold for fifth-grade 
teachers, however. // 

As shown in Exhibit i, most programs also require some sort of evaluation. 
Especially in ti^me of scarce resources, administrators ar^assessing the effects 
of their efforts or comparing different strategies for better effectiveness. 
Earlier research suggested that rational . decisionmaking about educational 
programs such as would use (^valuation studies for informatiori about t^^ \ 
efforts was rare (David, 1978). Later evidence, even though informal at this 
time, suggests that, indeed, districts with more effective educational programs 
do also have good evaluation strategies (Crandall, forthcoming). V7 

Much effort is underway to support evaluations leading to program impco^ement. 
Staff in ten regional Technical Assistance Centers focusing on evoJuation efforts 
in ESEA Title I districts report increased numbers of requests for help in the use 
of evaluation results for educational improvement. The development of 
materials and workshop curriculums dealing with this topic has beeh a major 
priority for two years. Again, there is no direct evidence yet of the pay-offs of 
all these efforts, but a study assessing the evaluation models and technical 
assistance work under Title I should provide some insight into this by no later 
than February 1982. 

Another program component believed to support the quality of education is , 
parent involvement. Occasionally, efforts to identify tjfxe components leading to 
program success have discovered more parent involvement in effective sites and 
less in ineffective sites (Wellisch, et al., 1976). In addition, a cur'rent study has 
described parent involvement in representative samples of sites for ESEA Title I, 
Ti|le Vn Bilingual, ESAA, and Follow Through in terms of acti]^ities undertaken 
by both councils and other parents and the numbers of parents involved in each. 



o 
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General findings have been that the content of legislation and regulations greatly 
influences what districts do with parents, the level of funding influences parent 
involvement, and monitoring paciental involvement activities is important. 
Specifically, greater involvement was found in instances in which specific 
activities were stated in the legislation or regulations, and there were incentives 
(such as priority for work as aides) for parents to be involved (Keesling, 1980). 



In summary, a body of evidence is accumulating about how program components 
are implemented by Federal education grantees and how those activities may 
lead to improvements in the quality of education. While one must continually 
examine questions of program targeting, services, and the V:?ffectiveness of 
Federal education programs, it is also important, especially as education laws 
are being analy:^ed and rewritten, to compile whatever data possible to address 
the separate components possibly influencing that effectiveness. As discussion 
of effective strategies and components progresses — as opposed to global notions 
of overall program, success or failure — Federal efforts to facilitate the 
improvement of education can be strengthened. 
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Postsecondary Education Programs 

The primary goal of the Department of Education's postsecondary education 
programs in fiscal year 1981 was to increase educational opportunity. The 
strategy for achieving this goal rested principally on providing financial 
assistance to students and to selected institutions. Federal student assistance is 
awarded both directly and indirectly through State a gencj /K and education 
institutions. In FY 1981, the obligations for ED's^^tsront financial assistance 
programs totaled almost $5.45 billion, ^^^stitutional assistance is granted 
primarily through the Developing Institutions Program and Special Programs for 
Disadvantaged Students. In fiscal year 1981, institutional assistance amounted 
to slightly more than $*00 million. 

Participation Rates 

Figures on college enrollment for the 18-24 year olds during the 1976-80 period 
are presented in Table 1. L' Participation rates over the rhpst recent 5-year 
period for the population as a whole indicate a continuous trend downward to 
1980 when enrollment rates reversed for both males and females, perhaps due in 
part to the impact of the Middle Income Student Assistance Act (MISAA). 1/ 
However, enrollment rates by themselves say nothing about how overall 
opportunities for college-age youth have been changing. For example,^ ^ny 
potential increase in enrollment figures for low-income or lminority students by 
increasing available financial aid may be offset entirely or partially by 
improvements in employment proispects for noncollege-educated individuals. 
Such a phenomenon would be most noticeable for students whose performance in 
college is predicted to .^e "marginal." Tables presented in this section should be 
interpreted with this in mind. 

Participation rates for the white subgroup indicated the same trend as the 
general population for both males and females. However, the rates for the non- 
white subgroups \collectively dii^played a consistent pattern; the rates for males 
fell in each caseVhUe the rates for females were consistent with the general 
population. Tabk 2 presents the annual enrollment rates of primary family 
members enrqlled in college from 1976-1979 by family income groups. Over this 
period, a clear trend of declining enrollment rates is evident in almost all income 
catisgories. However, while rates changed from two or three percentage points 
for families with incomes of less than $15,000, they dropped more than 8-10 
points for families in income categories above $15,000. The differences were 
not as volatile in nonwhite families but the patterns were similar. If 

Considering the reversal in the enrollment rates for 1980 and the increase in the 
level of benefits and the number of recipients, tliere does seem to be an active 
academic market place. Prior to 1980, falling enrollment rates suggested that 
the perceived long-'term financial benefits from a postsecondary education were 
less attractive than they once were. The addition of student funds through 
MISAAy however, has apparently reduced the net price to a level that again 
makes the investment in a postsecondary education attractive to a substantial 
portion of the 18-2<^ age group. The condition of the economy may also have had 
an influence upon the age group as larger numbers of students remain in school 
because of limited current employment opportunities. 
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•Kible 1 

Total Nunter lund Percentage of la* to-24- Year Olda 
Qir oiled In College, By Sex« Race^ and Spanish Origin (in IhouaaMa) 1976--80 

All • White Black Other Spaniah Origin 

Itotal Male PenBle Ototal Male ^^snale Ototal Male Fenale Ototal Male Penale Itotal Male Finale 

OODOGB) 76 

Pop ~~^919 13012 13907 23120 11279 Il8il0 3315 1503 1813 2-: 251 1551 701 850 

In College 7181 1673 350B f276 3250 3026 7'*8 331 ill7 157 92 65 310 150 I60 
% in college 26.7 28.2 25.2 27.1 28.8 25.6 22.6 22.0 23.0 32.1 ilO.O 25.6 20.0 21.1 18.8 

ocmfiot 77 

Pop 27311 13218 11113, 23110 11115 11995 3387 1528 1859 511 215 259 I609 . 751 855 

In College 7113 3712 3131 6209 "^3286 2923 772 309 113 212 117 95 278 139 139 

S in collego 26.1 28.1 21.3 26.5 28.7 21.1 21.3 20^2 22.2 11.2 I7.8 36.7 17.3 18.1 16.3 

QCVCXSR 78 

Pop 27617 13385 11262 23650 11572, 12078 3151 1551 1897 516 259 287 1672 78I 891 

In College 6991 3621 3373 6077 3195 1 2882 ^ 695 305 390 222 121 101 251 126 128 

$ in college 25.3 27.1 23.7 25.7 27 .6^^ 23.9 20.1 19.6 20.6 10.7 16.7 35.2 15.2 16.1 11.1 

79 

Pop 27971 13571.11103 23895 11721 1^1 3511 1577 1931 568 273 295 1751 337 917 

In College 6990 3508 31te 6II9 310^ joj^ 696 301 392 175 100 75 293 153 110 

% In college 25.O 25.8 21.2 25.6 26^ ^8 19.8 19.3 20.3 30.8 36.6 25.1 16.7 I8.3 15.3 

OOIDfiB) 80 • 

Pop 28130 13652 III78 23975 11767 12208 . 3555 I6OO 1955 600 285 315 1962 - 971 992 

In College 7226 36OI 3625 6331 3221 3110 688 278 110 ^ 201 102 105 315^ 151 I60 

% in college 25.7 26.1 25-0 26.1 27.1 25-5 19.1 17.1 21.G. 31.0 ^5-8 33-3 '16.1 15.9 l6;r 

* Students of other races are included %i cotal for all students but are not listed separately. 

Source: Calculated frcm U.S. Bureau of Census, Current Population Reports , Series^ P-20, No. 2^2, Table 11; 
No. 211« Table 13; No. 260, Table 12; Nob. ^72, 286» 303, 319> 333> 316, 360, teble 13. 
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FERCEirrAaE op 18-24 YEAR OLD PRXNARY FAKZLY MEMBERS £llRQLLiEZ) IN CX2UH3E, BY RACE, SE3C AND FAMILY INCOME 



All Races 



Tot. 



Uhlte 
M P 



Black 
Tbt. M F 



Tbt. 




Spanish Origin 



Tot 



M 



F 



' 1976 IxvBCtB^ 

B,000- 9.999 
10,000-14.999 
15,000-19.999 
20,000-24,999 
25.000 ♦ . 
All IncomsQ 



,14.5 
•l5.1 

21.4 

32.8^ 

44.1 

55.2 

26.7 



17.2 
17.1 
22.6 
33.1 
41.8 
52.9 
28.5 



2646 
12.6 
13.5 
20.4 
32.5 
46.4 
57.9 
25.1 



29.4 
13.8 
14.7 
20.0 
32.1 
43.6 
f5.3 
27.3 



30.0 

18. 3 

17.1 

21.4. 

32.2 

41.4 

53.1 

29.2 



28.8 
10.2 
12.7 
18.8 
. 32.0 
46.0 
58.1 
25.4 



10.6 
^ 15.9 
17.5 
32.2 
37.1 
58.1 
59.5 
22.9 



10.1 
14.3 
17.3 
31.9 
35.4 
52.1 
51.6 
22.5 



U.l 
16.8 
17.8 

32.5 
39.2 
64.4 
65.1 
23.2 



41.8 
15.2 
14.5 
32.4 
54.3 
31.4 
36.1 
29.7 



60.9 
15.6 
14.3 
19.2 
63.6 
33.3 
42.1 
34.1 



28.1 
14.7 
14.3 
39.6 
30.8 
30.4 
29.4 
26.0 



22.2 
12.1 
15.2 
25.2 
41.2 
34.3 
43.8 
20.4 



26.2 
9.4 
18. 9 
28.9 
39.6 
30.6 
40.0' 
22.4 



19.3 
13.7 
12.1 
22.8 
42.9 
38.7 
53.8 
18.9 



1977 Incorae^ 
II.R. 

0- 4.999 
5,000- 9,999 
10,000-14,999 
-15,000-19,999 
20,000-24,999 
25,000 -I* 
^All Incam 



28.2 
13.0 
15.5 
19.3 
26.4 
38.2 
54.6 
26.1 



30.2 

i4.i 

,17.4 . 
•20.4 
27.2 
i/0.1 

28.4 



26.3 

12.1 

144). 

18.4 

25.7 

36.3 

55.3 

24.1 



29.7 
12.4 
li>.0 
18.5 
25.9 
38.1 
54.6 
26.7 



31.7 

14.4 

16.7 

19.7 

26.9. 

40.5/ 

54.3 

29.3 



27.8 
10.8 
U.8 
17.4 
25.0 
35.5 
54.9 
24.3 



16.0 
13.7 
20.3 
25.2 
29.2 
36.1 
48.9 
?1.0 



15.4 
I3.J 
18.6 

25.3 
27.4 

29.3 
40.0 
19.9 



16.5 
13.9 
21.8 
25.0 
30.5 
40.3 
62.9 
21.8 



26.2 
24.1 
32.9 
29.2 
46.5 
51.4 
64.7 
36.0 



35.0 
30.8 
43.3 
25.0 
46.4 
41.2 
63.0 
40.9 



18.2 
18.8 
,26.8 
^32.5 
46.7 
61.1 
66.7 
31.7 



12.8 
10.0 
12.8 
15.8 
32.1 
36.5 
46.3 
16.9 



10.9 
12.6 
13.7 
16.7 
33.8 
36.4 
48.6 
18.2 



14.1 
8.3 
U.9 
15.2 
30.2 
36.6 
43.8 
15.9 



1978 Income, 

N.R.» 28.7 

0- 4,999 13.1 

5,000- 9.999, 15.1 

10,000-14,999 18.1 

15,000-19,999 23.3 

20,000-24,999 32.6 

25,00c + 48.1 

All In3cn« 25.6- 



1979 IncoL 

M.R. ^ 

4,999 

5i000- 9.999 

10,000-14,999^ 16.1 

15,000-19,-999 22.7 

20,000-24,999 28.2 

25.000 ♦ 45.6 

All Ineonw 25*2 



27. 
12. 

14. 



29.5 
16.6 
17.2 
19.6 
25.1 
32.9 
47.0 
"27.7 



28.0 
14.0 
14.6 
17.2 
24.1 
.26.4 
44.3 
26.3 



27.9 
10.8 
13.4 

^.7 
32.2 
49.6 
23.8 



26.2 

10.8 

14.8 

15.3, 

21slS 

30 .'0 

47.0 

24.2 



29.1 
12.6 
13.8 
17.8 
23.0 
32.3 
48.4 
26.2 



28.0 
U.5 
14.0 
.15.0 
22.1 
27.7 
45.7 
25.9 



30.0 
16.5 
16.2 
19.4 
25.2 
33i2 
47.4 
28.5 



28.9 
U.9 
14.4 
16.6 
23.3 
26.0 
44.6 
27.1 



28.3 
9.8 
11.8 
16.3 
21.1 
31.4 
49.6 
24.1 



26.8 
U.2 
13.6 
13>8 
21.1 
29.2 
47.0 
24.8 



2H.1 

13.7 
17.9 
21.5 

a.8 
34.9 

41.3 
20.4 



18.6 
12.2 
16.1 
17.5 
26.3 
32.4 
36.3 
19.9 



21.4 
16.2 
17.8 
19.6 
17.1 
25.3 
35.2 
19.7 



16.8 
17.7 
14.5 
17.2 
30.1 
24.6 
25.0 
19.0 



26.3 
12.2 
17.9 
23.1 
25.6 
46.3 
50.0 
20.9 



20.1 
10.3 
17.5 
23.0 
23.3 
41.3 
46.3 
20.6 



35.4 
18.4 
31.5 
29.7 
50.8 
39.0 
48.5 
36.8 



42,4 

I6.3 
22.4 
35.7 
32.8 
34.6 
6ls^l 
3l^S 



52.4 
26.1 
38.0 
32.1 
68.2 
42 .Q 
51.7 
^3.3. 



42.9 
26.1 
20.9 
45.5 
32.3 
54.2 
72.7 
37v.2 



22.2 
U.5 
'25.9 
27.8 
45.5 
35.5 
46.2 
31.2 



41.7 
7.7 
23.6 
29.4 
33.3 
17.9 
51.3 
25.8 



14.3 
8.5 
8.5 
18.2 
22.1 
25.4 
47.4 
14.9 



8.9 
15.6 
10.0 
15.7 
21.7 
19.5 
40.3 
16.4 



6.1 
n.3 

8.2 
20.4 
29.2 
25.6 

51.9 
16.2 



8.7 
18.0 

9.8 
17.5 
30.7 
18.6 
42.9 
19.1 



19.5 
6.5 
8.8 
16.2 
16.0 
25.0 
43.3 
13.9 



9.1 
13.8 
10.2 
14.1 
U.5 
20.5 
37.7 
14.1 



KIncaae not reported ' ^ , 

**udaed on differences. SnaU runbers in' Individual Income classes may cause large variations In percentage. 

SOURCE: Calculated from Q.S. Bureau of Census, Current Population Reports . Series ?-20, -\ ' 

Nos. 286,-303,(319. 333, 346. ?30. V^--^-^ 



The preceding discussion has focused exclusively on college enrollment) because 
no comparable time-series data exist for the non-<olleglate sector of 
postsecondary education — proprietary and public technlc.J/vocatlonal schools. 

' Improving Access and Choice Through Student Support 

J&ihanclng educational opportunity through student financial aid programs 
removes financial barriers to postsecondary education for qualified applicants 
(access)/ and provides each potential student with a*" wide a variety of 
postsecondary education options as possible (choice). A number of assumptions 
underlie this general posltloil. They are: (1) that parents bear the primary 
responsibility for financing their children's education; (2) that students 
themselves bear some of the burden of financing their education; (3) that the 
portion of the financial burden borne by the students themselves be distributed 
as equitably as possible; and (4) that the most n^dy students be aided first. 

ED-admlnlstered programs provide three types of student assistance. Grant aid 
"feon-retufnable) is provided by the Pell Grant, Supplemental Educational 
Opportunity Grant (SEOG), and State Student Incentive Grant (SSIG) programs.*' 
Obligations for these programs totaled almost $3 billion in fiscal year 1981. 
Loans are provided by the Guaranteed Student Loan (GSL), National Direct 
Student Loan (NDSL), and Health Education Assistance Loan (HEAL) programs. 
The 1981 obligations for these programs amounted to f»2.1 billion.1' Earnings are 
provided by the College Work-Study (CWS) Program, pbligations for CWS were 
again $550 million in fiscal year 1981. ^ 

The grant programs (as opposed to the loan and work ^programs) have distinct 
roles in removing financial barriers to access and increasing choice. Grants are 
seen as providing the financial support that less well-off parents are unable to 
provide. Loans and work-study jobs, on the other hand, offer students the means 
to share the financial burden of attending an educational institution. Offering 
the students both loan and work opportunities allows them a choice in the timing 
of when they ultimately t|ear their share of the financial burden. Work allows 
them to bear it concurrently with their schooling; loans allow them to put off the 
burden until after they finish their education. ) 

\ Impact of Student Financial Aid Programs , 
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The achievemej)t of the "access" objective of the student financial assistance 
programs can be measured by the absolute size of the financial burden a student 
must bear if he or she is to Attend a postsecondary institution. This "student 
burden" is the difference between ffie cost of attending an institution (tuition, 
fees, room, board, etc.) and th^ nonreturnable aid the student receives from his 
family^ the government, or other sources (i.e.; family contribution, grants, and 
scholarships). Presumably, thel student burden is financed by some combination 
of borrowing and work, and thus| for students attending postsecondary institutions 
it can be measured by the sum of their loans, earnings from part-time wprk,^^and 
savings from summer worl^ 

Two aspects of the student burden ^an be used in measuring the impact of 
student aid programs on "access." The first is the size of the student burden. If 
we assume that the student should be free to choose between current and future 
work, burden size should not exceed reasonable part-time earjfiings and summer 
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savings alone, or reasonable borrowing alone. The second important aspect of 
the student burden as a measure of program impact is the degree to which it is 
equal or unequal across heed categories, as measured by family resources, for 
different types of institutions (selectiveness, public or private, 2 years or 4 
years). This aspect may be interpreted as measuring the degree of equality of 
financial opportunity, which is derived from the premise that the burden borne 
by students should be as equal as possible. 

The performance of the student financial aid programs with respect to the 
"choice" objective can also be measured in terms of the student burden. Are all 
postsecondary education, options "affordable" in terms of the burden a student 
must bear, regardless of family resources? If burdens increase, can they still be 
covered by available loans and work? Relative income equality is a second 
useful criterion in evaluating fin^cial aid programs with respect to choice.^ The 
degree of equality in relative burdens serves as a measure of the extent to which 
grant programs equalise the financial terms upon which prospective students 
must choose between postsecondary options. Clearly, relative student burdens 
will b^equal if the student burden associatedowith each education option is the 
same rcigardless of the student's family resources*. An education option in this 
context can be defined by the total cost of attendance on the student's expense 
bi^dget. 

Evidence Concerning the Impact of ED's Student Financial Assistance Progrims 

Table 3 shows estimates of the mean expected family contribution, total grant 
aid received, and student burden borne by financial aid applicants, categorized 
by dependency status, family income for dependent students, and the cost for the 
institution attended. The estimates of mean student burden are useful in 
assessing the impact of ED's financial assistance programs. It should, however, 
be remembered that these estimates are derived from data on aid applicants 
only. 

A comparison of student burdens with what students can be reasonably expected 
to earn orl borrow sheds light on the achievement of both the access and choice 
objectives of the student financial aid programs. As a reference, a student 
working at the minimum wage for 15 hours a week during the school year could 
reasonably I expect to clear $1,200 for school use, while summer work could 
produce another $800 in savings, therefore, from work alone a student should be 
able to finance a student burden of $2!;0QJ9 provided, of course, that a job exists. 

^ • ''^ . , 

Under the Guaranteed Student Loan Program, all dependent undergradi ate 
students are eligible to borrow up ^o $2,300 per year to a maximum of $7,500; 
"needy" students can be allowed loans by postsecondary institutions of up to 
$6,000 in their first 2 years of study and another $3,000 in their second 2 years 
under the National Direct Student Loan Program. 

If access is interpreted as adequate financial resources to attend an institution 
with cost pf less than $4,000, the mean stuidbnt burdens presented in table 3, 
even if adjusted for inflation, indicate that the access objective has been 
reasonably acipomplished for aU.dependent students. For these students, the 
burden coUld be easily financed by part-time work alone. The independent 
students' burdens are significantly higher, but seem manageable if students 
commit much more time to work, or are willing to take out sizable loans.' 
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Mean Expected Far^ily Contrlbutlmy Grant Aid and Student Burden for 
Aid Applicants jy Dependency Status^ by Feuoally Inccroe, and by the 
Student Expense Pudget of the Instltuticn being Attended: 1979-1980 

(In dollars) ^ 



Dependent Students by Staily Inccme 



Student Expense 
Budget 

0 

$2,500 



.Expected Ftally Ccntrlbutlon a) 
All Grants 
Student Burden 



0 

-5.999 


$6»000 

.11^9? 


$12,000 
-17^??? 


$18,000 
^3.999 


$24,000 
-2?!??? 


$30,000 
and over 


241 


277 


625 


782 


1,253 


1,033 b) 


1,097- 


-.1,063 


829 


798 


. 680 


765 


510 


535 


574 


510" 


" 287 


250 




$2,501 - 


Expected Bteily Contribution ) 


439 


629 


661 


1,015 


1,171 


1,447 


29^ 


3,000 


All Grants 


1,230 


1,096 


1,082 


852 


694 


396 


7^ 




Student Burden 


1,049 


i.oa 


1,020 • 


869 


891 


808 


l,7p 


$3,001 - 


Expected Raunily Contribution a) 


302 


370 


^ 729 


1,151 


1,475 


1,645 


' ^5 


^1 4,000 


All Grants 


1,786 < 




1,473 


1,201 


823 


■ 1.018 \ 


^8 




Student Burden 


1,058 


1,573 


1,414 


1,229 


1,314 


959 


2,038 



N(4.001 - 


^cpected Ftaily Contribution a) 


257 


288 


767 


1.779 


2,665 


2,225 


■ 382 ' ^ ■ 


5,000 


All Grants 


2,260 


2,271 


1,880 


1,238 


752 


1,002 


j,288 




Student Burden 


2,037 


1,977 


1,902 




1,125 


1,348 


LsMl • 


$5,001 - 


Expected Family Contribution a) 


369 


545 


884 


-J 

1.405 


2,026 


3,lii6 


1 361 


6,000 


All Grants 


2,612 


2,405 


2,002 


1.904 


1,634 


. 979 


[1,424 




Student Burden 


2,437 


2,593 


\^2.,648 


2.251 


1,960 


1,396 


J 3,642 

1 : 


ymt - 


Q^peeted Family Contribution a) 


860 


823 


1.379 


2.450 


3,012 


4,738 1 


8?? 


^,000 


All Grants 


3,293 


3.366 




2.611 


2,256 


l.'*21 1 


1.515 




Student Burden 


3,222 


3,186 


3.413 


2,285 


2,202 


. 1.626 , 1 


5.349 



Note: a) Ihe Expected Amily Contribution is ccnputed according to the Uniform Itethodttlogy, adjusted where 
appropriate so as not tp be less than $1,110 for a student living with parents. 

b) Cell frequency fewer than 30* 

Source: **Study of the Intact of the Middle>*IncGae Student Assistance Act**, Applied Vlanagem^ Sciences, 
Silver Sjpring, HD, 1980. Contract funded by Office of Evaluatlion and Progrm Nanagenent/ED 
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What about choice? How feasible are the student burdens at private and 
proprietary higher cost institutions? Looking at table 3, we see that the largest 
mean student burden is $3,* 13 for dependent students. A burden of this 
magnitude is approximately $1,*00 more than the sum of reasonably expected 
student earnings ($2,000) and would require annijal student borrowing ($1,*13). 
Thus the student would have had to both borrow and work to meet the total cost 
of education. The largest me^ student burden is $5,3*9 (for independent 
students at. institutions costing over $6,000). A burden this large is over $800 
more than the sum of what a student can borrow and reasonably be expected to 
earn ($*,500). 

Table 3 also implies that, as institutional costs become greater, and given that 
the burden is a mean value, there, are large numbers of students whose burdens 
exceed allowable annual loan limits and reasonable levels of part-time earnings. 
Nonetheless, students still attend these institutions, making it reasonable to 
assume that students who are managing to enroll at more expensive Institutions 
could also have attended less expensive ones. On the basis of what is presented 
in this table, it seems safe to assume that a considerable degree of real choice 
among postsecondary institutions exists for most students. But for many 
students, choice comes only at the price of much higher personal sacrifice in 
terms of student burdens^ particularly for the independent student. However, as 
later discussion will show, MISAA may have had a profound impact on enhancing 
student choice. 

Turning to the question of whether there is equality in the financial terms^of 
access and! choice among students with different family resources, we can look 
at Table 3 again. A comparison of ' n ean student . burdens for differing 
dependency and family income categories, within institutional cost categories, 
indicates a high degree of equality almong dependent students whose family 
incomes are less than $2*,000. Mean- student burdens tend to decline for 
dependent students as they move to higher income categories. However, burdens 
increase markedly for both dependent and independent students as student 
budgets increase. 

In conclusion, it appears from the data available that ED*s student finaricial aid 
programs have been reasonably successful in providing financial access to some 
level of postsecondary education for all prospective students with financial need. 
These programs appear to have made even the most expensive postsecondary 
options potentially affordable -even for the poorest students. But attendance at 
high-cost institutions seems to come only at the price of very high personal 
sacrifice for certain categorief of ^tuderts, particularly the independent student. 

The Impact of the Middle Income Student Assistance Act 

The Middle Income Student Assistance Act (MISAA) became law on November 1, 
1978. However, its provisions affecting the BEOG, SEQG,^and CWS programs did 
not take effect until the start of the 1979-80 academic year. The intent of this 
legislation was to reduce the student burden of middle-income students. To see 
if this occurred, we can look at the average award and the proportion of 
recipients of any form of Federal financial assistance before and after enact- 
ment. Evidence from the Study of the Impact of the Middle Income Student 
Assistance Act indicates substantial y higher >ates of participation for 
sophomores and juniors and higher average awards, if Participation increased by 
almost * percent for sophomores and over 9 percent for the juniors while average 
size of the awards increased over 24 percent for both sophomores and juniors. . 
(See Table 4.) 



Table* 

Average Size of Total Award and Percentage Increase in Recipients 
of Any Form of Federal Financial Assistance for 
Sophomore and Junior Cohorts Before and After MISAA 



' Average Size of Percentage Increase 

Class Level Total Award in Recipients 

1978-79 1979-80 1978-79 to 1979-80 

Sophomores $1,*16 $1,760 

Juniors 51,50* $1,882 -9% 



Source: See Note 5/ ^ 

While MISAA provided for increased funding of all programs, the most significant 
increase in award level and participation rates was provided under the GSL 
program. Total loan funds available increased from almost $3 billion to $4.8 
billion as a result of the removal of the income test. The impact of this change 
is quite evident in Table 5 both in terms of awards and participation rates. 
Furthermore, overall funding for Basic Grants increased by less than $223 
million, resulting in larger numbers of upper- and middle-income students 
qualifying for the program. 

Table 5 provides detail on each program by income level and for 2 years. As 
noted«^ there were significant differences after MISAA was introduced. There 
was also* a significant reduc ion in NDSL recipients in the second year. 

Significant ^increases in the proportion of middle-and upper-income BEOG and 
G^L recipients took place. In addition, there were significant differences for the 
following groups: gain of low-income BEOG recipients; reduction of middle- 
income NDSL recipients; gain of upper -income CW5 recipients; gain of low- 
income GSL recipients. 

There were also significant gains in the average award in three of the four 
programs: BEOG, CWS, and GSL. The differences- in SEOG support were, not 
significant. 

The average BEOG gain was significantly greater for middle-income students 
than for lower*- and upper- income students. Also, lower-income CWS recipients 
gained significantly mor; than the other two income levels. None of the other 
levels was significantly different on any of the outcomes. 

Thus, MISAA appears to have had its intended effect with respect to channeling 
funds to middle- income students. 

Student Persistence 

As noted in earlier Annual Evaluation Reports, studies revealed that 70 percent 
of ail entering freshmen eventually graduate from^ college. Fifty percent of this 
group graduated from the institution at which they started; the others graduated 
after transferring to other institutions. Furthermore, 30 percent of the students 
who transferred did so in their second year. 

Analysis of recent data collected by an ED study on student persistence indicates^ 
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T«blt 5 

Avtrtgt Awird And Proportion Of Rtcfpltnts Across 
Fintncltl Asslstanct Progrw For Sophoaort And Junior 
Coliorts Irolwn OoMn 8y Studant IncoM Uvtl 

AVEMS AUAROS 



BE06 

Class Uvol and Studtnt Inam 1978-/9 1979^ 



SE06 
T978.79 1979-80 



HDSL 
1978-79 1979*W 



CWS 

1978-79 1979-W 



GSL 

1978-79 1979-80 



Sophoaort 



Low Incoat 


961 


1,110 


172 


196 


245 


215 


291 


350 


140 


178 


MIddU Incoat 


267 


672 


105 


154 


307 


265 


294 


320 


229 


332 


Uppor Incoao 


12 


266 


30 


61 


164 


236 


16V 


263 


593 


615 


r 

Junior 






















Low IncoM 


929 


1,149 


172 


214 


292 


249 


355 


362 


135 


213 


NIddU Iftcoa* 


249 


719 


120 


117 


266 


239 


276 


322 


362 


325 


Uppor Incoao 


26 


255 


29 


53 


185 


162 


158 


280 


593 





raOPORTIOII OF RECIPIENTS 
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Financial Aid Prograa 
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•04 
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.33 



Sourct: Stt nott 6 



23 




that financial aid is reined to student persistence in a positive manner. In 
particular, those students viH^ a grant or a loan above $1,000 were found to have 
higher persistence rates. Fmancial aid, as might be expected, assists the low- 
income student more than a student from higher-income families. Work aid 
appears to enhance the perslstenqe pf low-income students, especially if they 
receive no grant aid. Work aid, however, does not appear to be a factor for 
higher-income students, especially if they already have a grant. 

Further detail on student persistence must await completion of the National 
Longitudinal Study for 1980; however, the reason for the increase in 
participation rates noted in Table 1 could be as much a reflection of greater 
persistence as it is of the availability of MISAA funds (or both). 

The Delivery of Federal Student Assistance 

In the past, sufficient attention has not been given to the study of the delivery 
of Federal student aid funds. In large part this has been due to the ladk of a 
defined policy on the interrelationship of Federal programs, both in terms of 
award rules and the distribution of that assistance. During the past year, two 
studies brought to light the need for a more coordinated and coherent approach 
to the delivery of Federal student aid funds. This has become a high-priority 
concern in ED because of the increased size of both the Pell Grant and 
Guaranteed Loan Programs and the reduction in staff. Not only will an improved 
system be more ^effrCfent, but it can enhance the more equitable distribution of 
existing funds and reduce the opportunity for fraud, waste, and abuse. 

Improving AccesSt Choice, and Quality Through Support of Institutions 

The Department of Education also administers programs that grant funds 
directly to the institution. Programs such as these allow the institution to 
maintain and improve the quantity and quality of the educational services it 
provides to students without passing the full cost of these services on to 
students. These programs, like the student aid programs, also help the 
institution reduce or hold constant the burden to students in a period of rising 
qosts by reducing the net cost of instruction. 

Federal institutional aid programs administered by ED made up about 7 percent 
of the total ED higher education budget ($'>2^> million in fiscal year 1981 of a 
total of $5.7 billion). This amount included funds for the Special Services 
Program for Disadvantaged Students, Graduate Training programs, as well as the 
Developing Institutions Program (Title III of the Higher Educatfon Act, HEA) 
which accounted for slightly less than one-third of the funds for institutional 
support ($120 million in fiscal year 1981). HEA Title HI promotes detailed 
institutional program planning, curriculum development, faculty development, 
and improved administrative practices by providing resources to help 
participating institutions finance these prog^rams. Generally, the focus has been 
on institutions that serve large numbers of disadvantaged students. This focus is 
consistent with the larger goal of Federal aid for postsecondary programs: to 
enhance equal educational opportunity. HEA Title HI program eligibility criteria 
have included as a key factor the relative number of disadvantaged students 
whom the institution serves/ Thus, the program can be said to be on target if the 
relative number of disadvantaged served in developing institutions exceeds those 
in the student population as a whole. BEOG recipients, the economically 
disadvantaged students, on the average made up 28.5 percent of thie student body 
in HEA-Title III institutions compared to 17.9 percent for othfer 2-^ and ^f-year 
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colleges in academic year 1977-78. 

A review of participating institutions provides some evidence^ of program 
effectiveness. Few participating institutions have closed since the Inception of 
the Title HI program. Given that access and choice have been Federal goals, and 
that these institutions served large numbers of low-income students, the closing 
of any institution would reduce the number and choice of postsecondary 
education institutions available for students enrolled in the affected 
institutions.!/ Evidence shows that proximity to public and private non-selective 
colleges influences college entrance and that low-income students tend to live at 
home while attending college. The distance from home to college is strongly 
and consistently associated with parental income even after other student 
characteristics such as parental education, ability, and race have been taken into 
account, i Thus, certainly choice and, to a lesser degree, access are reduced 
with the closing of any institution. 

WhUe student financial assistancie programs can be judged in terms of their 
impact on either removing or equalizing financial barriers for students, other 
Federal programs have been aimed at assisting the student by removing pon- 
financial barriers to access and choice. Target groups for these programs have 
been disadvantaged persons with academic promise who lack adequate academic 
preparation or who are insufficiently motivated. 

These special programs attempt to identify potential students and provide them 
with counseling and remedial assistance to encourage them to enter and persist 
in postsecondary education. Present programs focus on students both at the pre- 
coUege and college levels; however, current funding levels allow for serving no 
more than one in seven students who might otherwise qualify. 

A 1979 study of the Upward Bound Program identified substantial impact upon 
students' entry into and persistence in postsecondary education programs. A 
recently completed study of the Special Services for Disadvantaged Students 
Program . indicated that students receiving the full range of this program's 
services were 2.26 times as likely to complete their freshman year as compared 
with similar students not receiving these services. They also attempted and 
completed more course units. , More detail on each study is presented in the 
program section of Volume II of this Annual Evaluation Report along with 
program and study information on the Talent Search and Educational Opportunity 
Center Programs. ' 

The Department of Education also administered a number of small categorical 
and grant aid programs of $30 million or less in annual appropriations. Included 
were fellowship programs, construction assistance, and international education. 
These programs are also discussed in detail in Volume 11. 
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NOTES 

// 

1/ Enrollment rates are for those in the 18-2* j^>ar old age group who are 
" currently enrolled. Previous reports hav^ included all those currently or 
ever enrolled. / 

// 

2/ College enrollment rates over time for various subpopulations (defined by 
" characteristics such as family income, sexJrace, and ethnicity) indicate 
changes in student pursuit of postsecondary schoo' opportunities. 

3/ Data were not available by family income for 1980; thus the impact of 
~ MISAA on enrollment rates by family income can not be described at this 
time. - " 

■ • ' ■.//•• 
y The State Student Incentive Grant (SSIQ) Program will not be included in 

" this discussion, since it provides matching grants to States to be used in 

their own student-^d programs and, as such, cannot be differentiated from 

State funds by recipients or financial aio administrators. 

5/ The $2.1 billion is the Federal subsidy /|for interest, defaults, administrative 
costs, and capital contributions for tl^ese programs. However, the actual 
value of loans made under the progra(rn totaled in excess of $3.3 billion, up 
from $3.7 billion in 1980. / 

6/ 4pP^^®^ Management Sciences, Stuciy of Program Management Procedures 
in the Campus Based and Basic Qrant Programs, MISAA Impact Analysis, 
Silver Spring, Md., September 1980. 

7f,^ This statement should be qualified in that the students may find other 

^ \ opportunities of equal or higher (juality. 

8/ Higher Education Research Institute, The Impact of Student Financial Aid 
" Programs on Student Choice, Draft Final Report, 1978. 
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special Category Programs 



\ Special Education Programs 

The ^veral programs authorized under the Education of the Handicapped Act (P.L. 
91-3a>) have had four basic purposes: (1) provision ordirect services; (2) development, 
demditJStP^^Wn^l^ and di§^ of n^^, technologies, teaching methods, and 

materials; (3) trainlfirig o^egular and special education personnel; and (*) program 
evaluation. In each program't^role of the Federal government has been as a stimulus 
in that it provided "seed" money^to^tates and other grantees to stimulate increased 
quantity and quality in all services ana'^^o-efisure,.4hat £rogram benefits reached 
previously unserved handicapped children. The strategy for evaRl^ting-programsjQ^^ 
the handicapped has been to determine whether they have accomplished their specific 
purposes and if they have had the desired stimulative effect. Accordingly, evaluation 
studies have been of two kinds: (1) those designed to obtain objective data on the 
impact and effectlvi^ness oi^spe^l|ied programs, giartiqularly tho^ eJivhigh Xiep r,esen^ 
major Federal investment of funds; and (2) those designed \6 provide policy-reievan;t 
planning information to enable the Federal government to target its resources more 
effectively. 

Studies of the first type found that, in general, these programs have accomplished 
their specific purposes. Efforts to isolate the stimulative effect and to demonstrate a 
causal relationship attributable to Federal programs have been complicated by factors 
^ outside the control of Federal evaluators and program managers. E\amples of these 
■^ factors range from effective lobbying by parents and professional groups with special 
interests in education of handicapped to court cases which have deiionstrated that 
handicapped children have not -had equal access to educational opportunities. Beacuse 
of these events,, an increase; in resources for handicapped children has led to a 
corresponding gradual increase in the number , of handicapped children receiving 
services. As noted above, the ^degree to which Federal programs hav^ contributeid^to ^ 
this increase has not been cieariy determined. Furthermore, attempts to demonstrate 
this effect were complicated by legislation which resulted in, a significant redefinition 
of the Federal role in education of the handicapped. 

The Education for All Handic^ped Children Act of 1975 (P.L. 9*-l*2) amended the 
Education of the Handicapped Act, Part B, in the following ways: 



o It explicitly stated i that Federal assistance to States was to ensure access 
to a free, appropriate public education for all handicapped children; 

o It specified that thp unserved had first priority, and that the most severely 
handicapped within each disability category who were not receiving an 
adequate education have second priority, for services relative to all 
handicapped children; 

o It specified that 'handicapped children should be served in the least 
restrictive environment consistent with their needs and abilities; 

o It specified that each child should have the benefit of an individualized 
educational program which would be updated at least annually. 



• ~ ' - " • .\ 

The Office of Special Education and Rehabilitation Services has data which show 
progress in achieving these legislative objectives. For example: 

o Almost 77 percent of the Nation's handicapped school-age children 
were receiving special education and related services in 1981 
compared to less than 50 percent at the time P.L. 9*-l*2 was 
enacted; ^ 

o In the annual child count for the school year 1979-80, *6 of the 5^ 
States and territories (79 percent) reported an increase over 1978-79 
in the number of handicapped children receiving special education 
and related services; 

o Since the passage of P.L. 94-1429 over 317,000 additional handicapped 
children have been reported to be recieiving special education which 
has required the hiring or reassignment of approximately 19,000 
teachers; 

o The number of preschool children ages 3 through 5 receiving special 
education has increased by more than 36,000 in the past 4 ye^ars, a 
growth of more than 10 percent; 

o Since the enactment of P.L. 94-142 in 1975, the number of 
ins: itutionalized handicapped children served by local districts has 
increased by almost 61 percent. 

Regional Resource Center Program 

The Regional Resource Center program was established to assist State and local 
education agencies in implementing the individualized education requirements 
and related services mandated by Public Law 94-142. 

The program was designed to improve educational services available to persons 
working with handicapped children by providing diversified resource services, 
improved educational methods, and training. In 1980, there were 15 centers with 
programs in all 50 States, the District of Columbia, U.S. Territories, and the 
Bureau of Indian Affairs schools. 

An evaluation of the Regional Resource Center progranri was completed in 1980. 
Thc^ objectives of the evaluation were (1) to analyze the procedures employed by 
the Regional Resource Center program in fulfilling the program goals; and (2) to 
assess the program^s impact on implementation of the P.L. 94-142 mandates 
regarding Individualized Education Programs, and free and appropriate public 
education for handicapped children. 

The study identifiec: a number of services that the Centiers provide. For 
example, Training is provided in the form oif workshops, conferences,, and 
inservice seminars; Demonstrations provide knowledge of models or 
methodologies which may be used in educating handicapped children. A third 
category. Other Assistancet includes such activities as planning and consultant 
services, dissemination and logistical support, and the design, development, and 
implementation of services, materials, or ( roducts. 



Some nriajor findings of the study showed that: 



o Program efforts in training and increasing the knowledge of P.L.. 9*- 
142 have b.een successful at both State and local, levels; 

o The RRC program was heavily involved in the development of 
products and materials for dissemination; 

o ThiB RRC services that have had the widest appeal and u^iUty^ave 
been training activities directed toward capacity building or the 
training of trainers at the State level. The predominant service 
delivery mechanisms have been consultation, dissemination, and 
/development of education products to meet State and local needs^^ 



Rehabilitation Services Administration 

The RehabiUtation^^^^S^^^ Administration (RS A) supports a wide variety of 
projectc and services to he^ disabled individuals achieve their potential for 
employment and independent living. ^ 

The major service delivery component under the Rehabilitation Act of 1973, as 
, amended, is the Basic State Grant program (Section 110). It provides funds (80 
percent Federal and 20 percent State matching funds) on a formula basis to State 
vocational rehabilitation agencies for physical and mental restorative services, 
training, counseling, reader services for the blind, job placement, technological 
aids, and other services. ^ 

Complementing the Basic State Grant program is a broad array of service 
projects designed to enhance the core vocational rehabili1;ation system (client 
assistance projects, jyojects for migratory workers, special projects for the 
severely handlcapped,\ projects with industry)?^ recently established programs 
which help complete the comprehensive service network (independent living 
projects, business opportunities, comprehensive rehabilitation centers); and 
mechanisms to maintain or improve the service delivery system (rehabilitation 
personnel training, technical assistance, interagency agreements, evaluation). 
The basic objective of the formula.and discretionary grant authorities is to build 
a comprehensive network of services that will eliminate major service gaps for a 
substantial portion of tuose who are most in need of services. This is meant to 
enhance their independence and contribution as members of the community. 



The Rehabilitation Services ^Administration has developed a comprehensive 
evaluation capacity for Federal and State vocational rehabilitation programis. 
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T^e purpose of the evaluation program is to: 

o Evaluate Federal forrrula grant programs and discretionary project 
V authorities in regard to their management and! outcomes; 

EvSuSion'^pwIof^ of the State vocaliional rehabilitatton 

program and projectl^authorities on the basis of RSA's genqral 
evaluation standards; — — 

Develop in State programs of vocational rehabilitation a capacity for 
evaluation which favorably affects policy development, management 
planning, and program operations. 

Evaluatiofi reports using the current national evaluatib^ standards are now ^ 
prepared for each State agency and regional office. These reports consider the 
status of major formula-grant program components and provide ci basis for 
monitoring and management initiatives. 

Comprehensive evaluation standards have been pretested, prior to national 
implementation in 1982, in the Vocational rehabilitation agencies in Delaware. 
Michigan, Mississippi, Oregon, Pennsylvania, and Virginia. The new standards i-se 
trend analysis, cause-effect angiysis, policy projections, and program simulation, 
for essential components of the formula-grant program and projects. 

The RehabiUtation .Services Administration^ has provided funding for the six 
States mentioned to develop model programs in evaluation to affect mahagement 
and policy ^development. . Successful results are now being disseminated to other 
State agenchfs trough technical assistance, publications and conferences. 

A number of research and evaluation studies of the^st 5 years have helped to * 
identify pew information needs in such areas as financial management, 
placement, client assistance projects, faciUties, evaluation standards, physical 
restoration ^services, and cost-benefit analysis. The project to develop a 
management information system was begun as a result of these earlier efforts. 
Utilization of data for management and policy a>ialysis was a special 
consideration in the design of the new system, which wiU be useful for simulation 
and technical forecasting of significant questions. 

National Institute of Handicapped Research 

The National Institute of Handicapped Research (NIHR) was created by the 1978 
Amendments (P.L. 95-602) to the RehabiUtation Act of 1973 (P.L. 93-112). Its 
purpose is to conduct a comprehensive research and development program that 
will improve services to handicapped persons and coordinate rehabilitation 
research Jbrqughout ifie Federal government. 

Major components of the NIHR resei'cfi^fSgrimrF^^^^ 

o Research and Training Centers (RTCs) that are mainly medical in. 
Emphasis but include work in the vocational, deafness, mental 
retardation, emotional illness, aging, and independent living areas. 
Eachf;has a^core area" of concern, and combines research, training, 
and liaisoi^w^th service a^nfij^es; 
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o Reflabilitatiotr&igineering Centers (RECs), which emphasize R<5tD to 
create and d^^lop new equipment and assistance devices to help 
rehabilitate disabled persons and make them more nearly 
independent; - 

• • . • ' ' • ..- 

o * Individual Research and Development (R&D) grant awards to develop 
^ "new knowledge about spinal cord injury, severe burns, head trauma, 

end-stage renal disease, and other specific medical and^ psy.chosocial 

problems of disabled persons; ' 

o A small international research and development effort, designed to 
find overseas innovations, usable in the United States, and a modest 
research-utilization and ii^fbrmation dissemination program, designed 
to promote adoption of NIHR research and development findings intb 
service programs, and generate knowledge on the processes of change 
and innovation. ' o 

Other activities in NIHR include:' 

o (Disseminating research and development findings to other Federal, 
* state, and local public agencies, and to private organizations engaged 
in rehabilitation; 

o Coordinating through an interagency Committee created by the 1^78. 
Amendments^ all Federal programs and policies on rehabilitation 
-^researchr^ In xloing^ this, NlHR wrki^ with and under the guidance of 
the National Council on the Handicapped, also established by the 
1978 Amendments; 

o Insofar as possible, educating'^^tlte^^blic on ways to rehabilitate 
disabled persons and improve family care aad self -care 



Seminating educational materials on rehabilitation to public 
schools, universities, and other public and private entities concerned 
with improving the quality of life for disabled persons; 

NIHR also produces and disseminates demographic reports on the total 
population of dibbled persons, gauge of needs and a guide to planning 
research and de^yelopment and setting other policies relating to disabled persons: 
This is done in cooperation \with other Federal agencies. Finally, NIHR develops 
and submits to the Congress and the President a long-range plan that identifies 
research to be done, funding priorities, and timetables: 
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Vocational Education Programs 



Vocational education Is a vast enterprise with a long-standing history of Federe 
support extending back to 1917. Vocational education legislation was 
traditionally a Vevenue-sharing program which provided. \ States and local 
education agencies with funds to supplement local resources. The 1976 
Amendments, however, provide detaiied, prescriptive requirements, for the 
expenditure of funds and for State planning, accountabiiity, and evaliiatlbn 
activities, as well. Current legislation provides funds for programs to prepare 
individuals for employment in the labor market and fo|r consumer and 
homemaking education. The Vocational Education Act also identifies persons 
with special needs aqd encourages including them in the mainstream of 
"vocational education by setting aside funds to meet their special needs. 

In fiscal year 1979, a^^tojal ^f 17.0 million studen^ were enroUed in state- 
administered vocatioyial education programs. Of these^^l 0.3 million were at the 
secondary level, 1.^ million at the postsecondary level, and 4.8 million at the 
adult level. « 

Vocaticibal Education programs are financed primarily by the States and 
localitlffi. h fiscal year 1979, the Federal contribution was 8.7 percent of a 
$6,479 bMlio^ program. Because of the large State and local overmatch, federal 
dollars nafj^^ stimulate the expenditure of State and local funds. Federal 
set-asldesfifiDr TlBtioilal priority groups, however, do have a catalytic effect. 5^t- 
asldes for the disadvantaged and handicapped ^have increased the number of 
programs for th^e groiips. 

^ . - - ^, • ■ 

Federal funding for vocational^pcograms is of two basic types: 

o Formula Grants to States . . ^ 

These grants are allocatied by formula to assist States to extend. 
Improve, and maintain exi^tj^g programs of vocational education so 
that persons of ^11 ages %111 have ready ai^^ess to high-quality 
vocational training or retraining. These grants^ alSo assist States to 
develop new progpao^ of vocational education and to provide part- 
time employmejrft f oh youths who^ need the* earnings from such 
emplpyn^^nt to contiijim^heir vocational training on ^ full-time basis. 
Funds are distributed through four separc^te/authorltlesr (1) Basic' 
Crants to States; (2) ProWams for Dlsadvarfuigedil (3) Consumer and 
Homemaking Education; ai^d (4) State Planning and Evaluation. The 
largest program is Basic Gi^ts which w^ funded at $518,139,000 in 
fiscal year 19|^f or use in 19^82. \ 

o~ Discretionary Pfc^raimsr 

Thi-ee programs award grants ^and/or contracts at the discretion of 
the Secretary: Programs <k National Significance (research, 
demonstration, and development projects); the Vocational Program 
for Indian Tribes and Organizations; and the Bilingual Vocational 
Training Program. \ 

\ ^ • \ 

Measurement problems and ambiguities in interpretation make it difficult to 
determine how successful V5)catlbnal education has been. One problcim is that 
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employment is not the primary objective of all vocational students. Many 
s^tudents take courses for avocational pursuits. Another is that economic 
cohditions probably much more powerfully influence employment choice among 
youth than does curriculum choice. Despite these limitations^ the data from four 
national longitudinal studies provide a sufficiently long time frame to asselss 
employment outcomes for vocational education graduates. \ 

o Taken together, the longitudinal studies suggest ihat most secondary 
vocational education graduates have no labor market advantages. 
The exceptiojis are young wom^n trained in the office occupations. 
Compared to their male counterparts and other vocational education 
graduates, they have fewer periods of unemployment, have higher 
hourly wages, and work in jobs related to their training. Other high 
school vocational graduates get jobs not too unlike nonvocational 
graduates within 3 years after graduating, have similar rates of pay 
and unemployment, have similar degrees of job satisfaiition, and are 
no more knowledgeable about occupations than are nonvocational 
students. Vocational education students are as likely to drop out as 
other students, are less likely than other graduates to continue 
formal education in colleges and universities, bat will probably seek 
additional occupational training. Postschool training (outside of 
colleges) pays o|f more for vocational graduates than for othjer 
graduates. « . 

Because secondary vocational students have lower aptitude test scores and tend 
to come from poorer families than other students, it is often assumed that these 
students would lose interest in school if vocational education were not available. 
This hypothesis, however, has never been tested. 

There are other studies which present different data and lead to different 
conclusions. The Office of Vocational and Adult Education (OVAE) has recent 
data from an OVAE - funded research contractor (the National Center for 
Research in Vocational Education, Ohio State University). the. basis of those 
data, the National Center has reached the conclusions presentied in the following 
paragraphs. 

' . ..' ^ ■ * 

OVAE has presented these findings with the general caveat that they reflect 
broad geni^ralizations that ignore mfany qualifications and exceptions, and that it 
is extremely difficult to generalize about vocational education because the 
differences across program areas are often as significant as the differences 
between vocational education and the other curricula, (the sources 'underlying 
these statiements are referenced by the number;^ in parentheses and listed below.) 

1. Vocational education assists students in the transition from school to work. 

o Secondary vocational graduates are more likely to enter the labor 
market and to be employed than graduates of other curricula. 
Unemployment rates do not differ significantly across curricula 
because other graduates are4ess likely to seek employment (2,4). 

o Secondary vocational graduates obtain regular, full-time jobs more 
quickly than graduates of other curricula W. 



Over half 
postsecondary 
related to 



their 



(usually closer to 70 /percent) of secondary and 
vocationsU graduates oj^ain jote In occupatidnal areas 
fields of trainirig (2,/^ 



2. 
3. 



o Secondary vp^cational graduates /are more likely than graduates of 
other curriqlUla to be assist^o by school personnel in j obtaining 
employment (3,*). 

o Placemj^t rates are highe/ in schools where sta^^ipjgrnbers agree on, 
the inr^ortance of placement (3). }. 

Vocation^ education reduces the risk of unemployment ibr mem of 
minorit]^groups (1,5). " y' ' X?.. : . ' 

The evidence on the effects of vocational education on earnings is mixed. 
Some studiessindicate 4n initial advantage that disappears within five to six 
years. Other^studies find no significant differences. Evidencpe on longer 
term effects/(s Umited and also mixed (2y4y5). | 

Employe^;/ are^^tisfied with the performance of vocational graduateSi and 
^egpaduates are satisfied with the preparation they received (2). 

There is insufficient evidence to judge the effects of vocational education 
upon the attainment of basic communication and computation skills. 
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o On standardized achievement tests secondary vocational students 
perform about the same as those in the general cui riculum and 
significantly below those in the academic (college preparatory) 
curriculum (2). ^ 

o About one-third of secondary vocational graduates continue their 
education beyond high school. This is slightly below the rates for 
general graduates and considerably' below academic graduates (2). 

There are a few indications that vocational education may retain potential 
dropouts, but the evidence is insufficient for a firm judgment (2,5). 



Sources of data from the National Center for Research in Vocational Education 

1. Darcy, Robert L. Some Key Outcomes of Vocational Education^ 
Coliimbus, Ohi^: National Center for Research in Vocatiornl 1 
Education, Ohio State University, 1980. r 

2. Mertens, Donna M., et al. The Effects of Participating in \ 
Vocational Education. C-*lumbus, Ohio: National Center for {Research in 
Vocatioi^al Education, Ohio State UnWersity, 1980. 

3. Preliminary analyses of data collected for the task on " Factors Affecting 
Job Placement in Vocational Education," National Center for Research in 
Vocational Education, Ohio State University. t 
Preliminary" analyses of a national survey conducted for the task pn "The 
Effects of Secondary Vocational Education on th^ Occupational Attainment 
of Younger Adult Workers," National Center/for Research in Vocational 
Education, Ohio State University. — 

4'j 
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5. Preliminary analysis of data from the new youth cohort of the Nationjal 
Longitudin^ Surveys for the task on "Patterns of • Participation in 
Vocational Education and Their Subsequent CXitcomes/' National Center for. 
Research in Vocational Education, Ohio State University, 

Community Education Programs 

The Community Schools Act of 197* indicated that the school,^ as the prime., 
educational institution of the community, is most effective when the school 
involves the people of the community in a program designed to fulfill their 
educational needs. To implement legislative mandates set forth in the Act, 
community education programs utilize schools and other public buildings as 
community centers. These centers provide education, recreational, cultural, and 
other related community services. Federal grants are provided to State and 
local education agencies to pay the Federal share of the cost of establishing and 
operating these programs. 

A national evaluation of the community education programs, performed under 
the direction of the Community Education Advisory Council, was completed in 
February 198l.| 

The purpose of the evaluation was: 

1 \ * 

1) To analyze the operations of State education agencies as they 
develop community education programs; 

2) To assess th^ impact of Federal sup).ort on the capabilities of State 
educatio\i agencies to develop leadership in the community education 
area. \ 

Findings showed that: ^ 

1) The States have been successful in developing State community 
< education systems capable of expanding and supporting community 

education programs; - 

2) . State governments have broadly expanded their" role as leaders in 

community education subsequent to the passage of the 1974 
Community Schools Act. In 1974, only nine States funded a 
community education position. Today, all States and the District of 
Columbia designate officials for community education activities. 

The study further showed that State needs assessments and interagency 
cooperation were the strongest operational elements of the programs; State 
planning, evaluation, and reporting were the weakest Operational elements. 

An important measuremient tool from the st^dy was the Community Education 
Development Index (CEDI). The CEDI identified common elements of State 
community education systems, and was useful in isolating factors which appeared 
to contribute to the development of State community education systems. 
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Highlights of Evaluation Results 



Evaluations assist Departn:)ent of Education officials as Well as members of the 
Congress in making informed decisions for improving ED program efficiency, 
effectiveness, and responsiveness. They attest to program successes and 
failures, strengths and weaknesses, and thus provide the primary source of 
objective evidence , used in determining future program operation and policy 
options. 

Evaluation results are widely used in the tDepartment of Education for budget 
preparation for ongoing programs and programs whose operations are changing, 
for budget hearings, for Congressional testimony, and for Departmental 
responses to Congressional inquiries; They provide ^background for the 
preparation of pplicy papers which affect ED-related legislation as \yell as for 
the writing of regulations which clarify how ED programs function. A base of 
information is provided by evaluation results which aids in decisions regarding 
the distribution of Federal funds to local areas as well as decisions which 
improve services to constituents:. 

The following section contains highlights of evaluation studies completed in 
fiscal year 1981. The studies are categorized under the headings of Elementary 
and Secondary Education, Postsecondary Education, and Special Category 
Programs. ^ 

Elementary and Secondary Education 

♦ Study of Evaluation Practices and Procedures 

Last year's Annual Evaluation Report (Fiscal Year 1980, Volume I) presented 
highlights of findings from the Study of Evaluation Practices and Procedures 
conducted by a Northwestern University team under Boruch and Cordray. That 
study responded to the "Holzman mandate" in the Education Amendments of 
1978 (Section 1526 of Public Uw 95-561). , 

During fiscal year 1981, a second and separate study in response to the same 
mandate was completed by the National Academy of Science's Committee on 
Program Evaluation in Education, under Raizen and Rossi. Two major findings 
characterized the Committee's Report: y 

o Evaluation must be viewed as a system that involves many 
organizations and parties- Attempts to improve the quality of 
1 evaluation studies or to ensure the use of evaluation results must deal 

with systemic problems rather than with the specific shortcomings of 
any ihdividual evaluation. 

. /" ■ - ^ 

o Both the quality and » the use of evaluations could be considerably 
Chanced through better management procedures. This is the -most 
important step that the Congress and the Department could take to 
increase quality and to promote better use of evaluation results. 

As had been the case in the Northwestern University report, the Committee on 
Program Evaluation in Education made one set of recommendations for the 
Congress and one set for the Department. 
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♦ Effectiveness of Title I of the Elementary and Secondary Education Act 

A ^-y ear study of programs under Title I of the Elementary and Secondary Education 
Act <P.L. 95-560 is nearing completion. It is; widely known as the "Sustaining Effects" 
Study. Mandated by Siection 183 of Title I, which reiquires the .Department to -conduct 
independent evaluations "which describe I and measure the impact of programs... 
assisted under this title",, the Sustaining Effects Study was designed to document the 
following; (1)" the characteristics of districts, sciiools and students participating in 
Title I; (2) the nature of Title I services provided to program participants; and (3) an 
analysis of the effectiveness of those services over several ye?.:s of program 
participation. 

The Sustaining Effects Study represents the most comprehensive effort to date to 
document and analyze the nature and effects of compensatory education programs. 
Dafta were collected on all students in a nationally^representative sample of over 200 
elementary schools during the 3-year period from the fall of 1975 through the spring of 
lSr78. Not only were data collected on student, teacher, and principal characteristics, 
and student progress in the basic skills and in .attitude toward school, but interviews 
wfere also conducted with parents of a representative subsample of 15,000 students. 
Information on the characteristics of participants and nonparticipants has been 
previously published in two Executive Summaries: "Elementary Schools and the 
Receipt of Compensatory Funds" (Mayeske, 1977); and "Student '^conomic Background, 
Achievement Status, and Selection for Compensatory Services" (Mayeske, 1978). 



The major findings to date of the Sustaining Effects Study include the following: 

o Of all Title/ 1 program participants (approximately 5 A million children, 
spanning 68% of the nation's schools), about 82% receive reading 
instruction and 3*% receive math. The level of instructional resources, 
devoted to Title I participants is about 1.5 times greater than for similar 
non-compensatory students. More than half the schools in the survey 
provided Title I services to students in private schools, with a comparable 
per-pupil expenditure. 



Compared to similarly needy but non-compensatory education students in 
economically deprived ar^as, the Title I students learned more in reading 
for grades 1-3 (though not significantly more in grades * - 6); in math- 
ematics, the Title I students significantly surpassed the non-Title I students 
in each grade from 1-6. 
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Table .6 

, Achievement. Impact — from the "Sustaining Effects Study," 

. • . . . . 

1976-77 data\ £roi& the Interim Report (1981) : 



READING GAINS (Standard Scores iV 

% Additional^ 



CE Group 



Non-CE 
Group 



MATHEMATICS GAINS 



CE Group 



Non-CE 
Group 



« Additional 
growth 



GRADE 



1 . 


61 


52 


17 % 


63 


2 


44 


40 , 


10 


56 


3 


34 


29 


13 


58 


4 


33 


31 


6 


55 


S 


26 


24 


8 


42 


6 


25 


25 


0 


47 



47 
48 
53 
47 
34 
27 



33 
17 
9 
17 
24 
74 



CE — conqpensatory education group 

lSk>n-OB — non-compensatory education group 



^Aaln is expressed in standard score units derived for the specially norjned 
versiondf. the California Test of Basic Skills used in this study. 

J^s "percent additional growth" measure represents the academic growth 
made by the 5 group above and bey6nd that made by the nonrCE group. 



o The achievement gains maofe by Title I students tended to persist 
over the summer and througH^^a subsequent school year even after 
services had been discontinued.Xwhile an earlier study suggested that 
. Title I students (in selected sitesi who did not attend summer school 
. sho>yed considerable losses, the Vesu Its of the Sustaining Ei^iects 
Study show that, on the average, tosses tend hot to occUr over the . 
summer for compensatory reading students. 

o Of those who receive Title I in any oneNyear, about *0% will not be in 
the program the following year; most of Nihese students (60% of those 
discontinued) "graduate out" due to highXachievement. The average 
percentile for those continuing in Title \ is 22, white the average 
percentile of those "graduated out" is After a year ^without 
services, children who "graduated out" tend to maintain their 
achievement at this higher level. 

o The factors found to be related to increased student achievement 
during the first year of the study include: greater amounts of regular^ 
and tutored instruction, greater teacher experience, lack n^fs^^^ 
disruptions to instruction, frequent feedback on progress, and greater ( 
teacher effort in planning and evaluation. However, there are not 
many strong relationships between the magnitude of the improvement 
and these other factors., ' , 

1979-80 was the first school year in which all States and local school districts 
participated in\the implementation of the Title I Evaluation and Reporting 
System (TIERS)\ the system of models and reporting forms designed to yield- 
aggregdtable, nationwide information about Title I participation and effects. As 
of September ll 1981, the Department of Education has received complete 
reports f rpm all State Education agencies. 

o ' Preliminary analyses of this information suggest that Title I programs 
in reading and mathematics can often be successful in raising the 
achievement levels of participants beyond what would be expected 
had they not received, compensatory instruction. Some results of 
these locally conducted evaluations are summarized in Tables 7 and 
8. 
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Table 7 

Achievement Inpact ; txam the Title I Evaluation and Reporting System, 
1979-80 (50 States and the District of Columbia) — Annual Testing 







NiSBber 


Percentiles 


% Additional 


Grade 


Tested 


Pretest 


Posttest 


growth 


BEADING 


2 


63,109 


28 


29 


4,»% . 




3 


78,452 


23 


26 


' 16 


(Annual Testing) 


4 


.78,832 


23 


26 


17 




S 


79,731 


22 


26 


20 




6 


75,396 


22 


27 


37 




7 


44,375 


22 


25 


. 22 




8. 


38,894 


22 


25 


23 


MATHEMATICS 


2 


39,008 


35 


" 37 


5 




3 


49,451 


31 


32 


3 


( Annual Testing) 


4 


52,203 


28 


30 


16 




5 


51,364 


26 


30 


22 




6 


50,496 


24 


31 


44 




7 


25,075 


23 


26 


27 




8 


21,196 


23 


27 


30 
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Table 8 

Achievement Impact — -from the Title I Evaluation and Reporting System, 
1979-80 ^30 States and the District of Columbia) ~ Fall-Spring Testing 



READING 



(Fiall-Springjfv 
Testing) . 



MATHEMATICS 

(Pall-Spring 
Testing^ 





Number 


Percentiles 


% Additional 


Grade 


Tested. 


Pretest 


Posttest 


^rovrth 


2 


208 > 101 


18 


32 


67 


3" • 


198,743 


16 


26 


92 


4 


183,500 


16 


25 


118 


■5- 


168,233 


16 


25 


145 


6 


147,247 


17 


25 


154 


7 


113,211 


16 


23 


149 


8 


90,863 


16 


22 


132 


2 


85,134 


20 


36 


74 . 


3 '■ 


96,336 


19 


32 


106 


4 


100,379 


18 


31 


153 


5 


92,650 


18 


• 30 


194 


6 


81,059 ' 


18 


29 


198 


7 


56,621 


18 


27 


171 


8 


46,231 


17 . 


26 


164 ^ 



-We results obtained from fall-spring testing seem to over-estimate the 
"real" iap^ct of Title 1 programs, due to methodological problems in the 
fall-spring data, «.g. , pretest scores are consistently too low. The results 
from annual testing <i.e. once-a-year testing) are mrre realistic as 
measures of the lasting academic growth of Title 1 participants, and 
the annual impact evaluationii more closely resemble the results obtained 
from national studies, such as the Sustaining Effects Study. The 
fall-spring results (which may accurately portray gains made during the 
course of the school year, rather than results which persist across years) 
are included here for conpleteness rather than is an accurate estimate of 
the effectiveness of Title I projects. 
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The results from national and from locally conducted evaluations suggest that a 
fifteen year decline in educational achievement is beginnin^^ to reverse, particularly 
among traditionally low-achievvng jgroups. For instance: \, 

o The National Assessment of . Educational Progress has document 
improvements in the educational status of minority-group nine-year, olds 
over the past four years, and has also sho wn improved achievement leyels 
in title I schools. p.. At least partially, these achievement gains may be 
attributable to increased atterjtion to basic skills and to effective 
compensatory education programs. , 



o In a sample of 16 New York City Community School Distric^C^der a new 
competency testing program that stressed reading comprehension, gains 
improve<| by 50% in comprehension and by 16% in vocabulary between 1980 
and 1981. At the same time, citywide test sicores as well as the Scholastic 
Aptitude Test results for New York State also exhibited gainsi surpassing 
national averajges for the first time in years* 

o In each of. the past two years, the typical pupil in Chicago Title I classes 
improved in reading by at least * percentile points, as measure^ by 
standardized tests. Some schools did even better with gains in some 
classes 67% higher than- those of similar students^ in the appropriate noun- 
groups. In addition, 80% of the parents of these Title I children concurred 
wi^h involved teachers and principals in endorsing the Title I projects. 

o In New Jersey^ Basic Skills Improvement (BSD programs represent/ 
com]pensatory education programs funded through combinations of ESEA/ 
Title\l, State compensatory education, and school district efforts\i New/ 
Jersey reported that not only had the programs accelerated the acquisition! 
of basic skills, but that the acceleration was greater in 198Q than it has 
been since the State first began evaluating its programs.. In reading, BS(I 
students improved their performance from an average pretest mean at thfe 
23rd percentile to a posttest mean at the 36th percentile. In mathematics, 
students moved from the 26th percentile at the beginning of the program 
to the *2nd percentile in the following spring. 

♦ The ESiEA Title I Evaluation and Reporting System 

Evaluation and reporting have been requirements of Title I of the Elementary ^nd 
Secondary Education Act since the inception of the law io 1965. Early attempts to 
•synthesize and consolidate information about the program, however, based on S^ate 
evaluation reports, proved impossible. Not only did the quality , and thoroughnesis of 
the evaluation reports vary tremendously, but the types of information presented 
ranged from complete participation and effectiveness data down to th^^esentati^n of 
only a few case studies and testimonial pieces. ^ 

Frustrated by the perennial lack of nationwide information, th^e Congress used the 
Education Amendments of 197* to rework completely the evaluation Pequirements of 
local and State education agencies. Specifically, the Congress added requirements 
that the then - U.S. Office 6iE Education publish standards and uniform criteria for 
evaluation, develop models that can provide comparable ;^nf or mation on the 
effectiveness of projfects, provide technical assistance to Statf and local school 
personnel to. assist them in performing their evaluations, and report periodically to the 
Congress. (ESEA, Title I, Section 183.) • i 



in 1976, the Office of Education began the long process of implementing a new set of 
evaluation inodels, suppored by the newly initiated Technical Assistance Centers 
(TAC's). Even though the new evaluation requirements were not published in the 
Federal Register until October 12, 1979, as .*arly as school year 1976-77 about 20 
States: implemented the new models on a pilot basis. Jn school year 1977-78, virtually 
every State, ha^ identified school districts willing to try out one of the proposed 
evaluation models, ^d several States were already using the.-new system statewide. • 
School year 1978-79 saw most States not only fully implementing the Title ! evaluation 
and Reporting System (TIERS), but working with ED and the TACs to improve 
generally their procedures for selecting, admini'stering, scoring and analyzing tests, to 
pei form district rieedf> assessmehts, and to select appropriate children for 
participation in cornjjensatory education programs. 

« «* 

Nationwide implementation of the TIERS-^ccurred in the 1979-80 school year. 
Reports submitted t^y each State to ED were due in February 1981, and ED will repdft 
to the Congress on the nature and effectiveness of Title I by February 1, 1982. The 
State reports will Qbnstitute a major source of information fqjr the mandated report to 
the Congress, though substantial support will be provided through other studies in the 
Off ice of Planning,^ Budget and Evaluation, most notably the Study of the Sustaining 
Effects of Compensatory Education. 

Highlights of early implementation of the TIERS are as Jollows: ^ ^ 

o As of July 1, 1981, (l5 States had submitted completed Title I participation 
and evaluation reports. ED anticipated that all States would ^omplete 
their reports by August I, allowing the mandated report to the Congress to 
be based upon complete national data. In previous years, no moreVh^ *0 
States, and frequently fewer, provided comparable annual information on 
Title I program services and evaluations to the Department. 

o Many \ States successfully implem^ted extensive "quality cdntrol" 
procedures for reviewing, editing, ana correcting evaluation infornlation . 
submitted by local school districts. Ma^ States ha^e also begun prowding 
descriptive information and information on achievement gains bacRsJo 
participating school districts on their status relative to similar district^ 
States! are also providing districts with information on how their evaluation 
procedures can be strengtheried. 

o .There I is a nationwide focus on identifying and sharing information about 
exemplary programs. Often with the help of the TAC's, States are 
coordiiiating activities with ED'S Office of Compensatory Education to 
identify and then assist school districts. mtj[i unusually effective 
educational programs, in order to prepare and present subr^^ to the 
Depariment's Joint Dissemination Review Panel (JpRP). Projects 
validajted as exemplary by the 3DRR are described annually in the ED 
publicktion Educational Programs That. Work. These projects are often 
used ais models for other projects with similar academic areas, and become 
eligible for funding by the National Diffusion Network. 

Preliminary results from the implementation of the TIERS models seem to 
indicate that Title I programs in reading and in mathematics are often 
successful in moderately raising the achievement levels of participants 
above what would h;*ve been expected without compensatory educational 
assistance. These finding^, while tentative pending further analysis (to^be 

\6 



fully discussed in the mandated report to the Congress on Title I), are 
supported by results of other studies as well, particularly the Study ol the 
Sustaining Effects of Compensatory Education and by recent longitudinal 
analyses of the National Assessfment of Educational Progress. 

For reading and mathematics projects in grades 2, 6, and 10, States have 
provided ED with detailed information on program characteristics, ED will 
be examining iri detail the distributions of various instructional strategies, 
allocated riesources, student-teacher ratios, and other background variables 
and their interrelationships. In addition, extensive data w^n evaluation 
model implementation, test selection, and summer effects will be 

examined in relationship to achievement gains. 

• • , .■■ 

States and Ipcal school districts have been avid consumers of TAC services. 
In fiscal year i 980, ^in addition to continued assistance in evaluation model 
implementation, iihterpi^etation and quality control, the TAC's emphasized 
training activities that would lead to improved methods for selecting 
participants and for \conducting needs assessments, a'^.d evaluation 
procedures that could\ be used to identify program strengths and 
weaknesses. During the i$-month period from October 1, 198G to March 31, 
1981, the TAC's provided about 900 workshops and 1,100 on-site 
consultations, ail at the request of State and local personnel. 
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♦ State Refinements to the P.SEA Title I Evaluation and Reporting System 

In an attempt to increase the relevance of Title revaluation data for school 
practices, ED has sponsored several projects in "State Refinements 1o the ESEA 
Title I Evaluation and Reporting System." 

The purpose of these efforts has been to support further developmental work by 
State education agencies (and by school districts in conjunction with their Stat^ 
agency) in response to subsections (c) and (e) of Section 183 of the Elementary 
and Secondary Education Act (P.L. 89-10, as amended by P.L. 95-561). These 
subsections called -for "jointly sponsored objective evaluation studies of 
programs and projects assisted under this title"" and for "technical and other 
assistance as may be necessary to State educational agencies to enable them to 
assist local educational agencies and State agencies in the development and 
application of a systematic evaluation of programs". 

Fourteen studies were sponsored in fiscal year 1979 and these efforts can be 
grouped into two main categories: 1) improvement of datS collection and 
analysis activities, and 2) projects related to testing and evaluation methodology. 
The first category included a study by the State of Pennsylvania to determine 
the types of error being made in Title I data collection and analysis and to 
develop materials to reduce the number and severity of these errorSi The final 
report detscribes the various types of errors made during Title I evaluations?, and 
contains a FORTRAN program (using the computerized norms of eight major 
Title I tests ) which allows for the conversion and analysis, at the State level, of 
all local district test scores. The State of Arkansas developed user-oriented 
instructional materials to reduce the number of errors in Title I evaluation 
.reporting. Components of the Arkansas ieffort include three filmstrips, four 
audiotapes, and a detailed program med-text handbook on implementation 
procedures for Model A of the Title I Evaluation and Reporting System. 

In the second year of the State Refinement effort (fiscal year 1980), ED added 
two additional categories for study. One category focused on improving the use 
of evaluation information at the local and State agency levels, and the second 
examined local management efficiency related to the adoption of the Title I 
basic skills models. For example, in the first category, the State of New York 
has developed $ computer-based feedback system for use in reporting Title I 
evaluation results back to local distrjpl§.^ Compatible with New York's 
previously existing individual student data system, the newly developed feedback 
system Ccin be used, at the local district level, to link formative and summative 
evaluation efforts, by displaying Title I student achievement against allocated 
ii^tructiohal resources. 

The second category involved efforts to investigate components^ of the actual 
operatiofis of Title I evaluations so as to improve the coordination of Title I 
evaluation with other district activities (e.g., a review and analysis of how state 
agency or school district testing programs could be coordinated with local and 
Federal program evaluations). The State of Wisconsin developed a framework of 
program characteristics so that evaluation data from similar districts can be 
compared* 
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♦ Assessin|^th.§^^Effectiveness of Current and Alternative i Comparability 
Provisions ' ' 

A study was conducted to assess the effectiveness of thie current comparability 
provisions in ESEA Title I, and the feasibility and desirability of alternative 
provisions which might offer greater flexibility to school districts without 
compromising the purpose of the Federal comparability provisions. The study 
was mandated by the Congress as an opportunity for districts to use different 
measures of comparability to allocate resources for the 1980-81 school year. 

Five hundred districts selected through a stratified random sample were 
contacted by telephone to determine interest in participating in this study. 
Sixty-one percent indicated no problems with current provisions, 33 percent 
indicated ^problems related to conflict, or burden, or both, and 7 percent 
indicated some problem that might be related to comparability. Only 13 percent 
of the initial 500 districts were interested in. participating in the study, and 9 
percent did so. Of the districts in the study, 95 percent identified at least 
one comparability task as burdensome under current comparability provisions, 
but only 16 percent of the districts citing '"burdeh" as a problem spent over 60 
person days on comparability. Seventy-five percent of the participants reported 
that comparability conflicted with State and local goals for allocating staff, 
programs, and services. However, only one district provided a concrete example 
of how the current comparability provisions prevented it from carrying oit local 
policies , in allocating staff and services to schools. 

Most of the alternatives sel^^d- by school districts modified the current 
comparability criteria (pupils peb^nstruct staff, and expenditures for 
instructional staff salaries per pupiiKby eliminatijig one dftthe ratios and/or 
changing the definition of the criteria. "-Ij was found that changes to current 
criteria would eliminate tasks associated^^rith. data collection which many 
districts consider t6 be burdensome. Howev^ the reduction in the level of 
effort required of district staff\would be modest. Changes to the reporting 
requirements for the preparation, of revised reports and maintenance of 
comparability would significantly reduce burden in districts with high staff and 
pupil turnover. The likelihood that* Title I schools would receive fewer resources 
than non-Title I schools would increase, tiowever. 

Most of the alternatives implemented ^^d^iittle impact on the conflicts reported 
by schools, primarily because they wereXdesigned ^to reduce administrative 
burden. \ 

♦ Examining the Effort to Reduce Disproportionate Disci[5linary Actions Against 
Minority Students \ 

A recent study of the Emergency School Aid Act\ESAA) program to reduce 
-disproportionate disciplinary actions against minority students describes a 
sample of 15 ESAA projects (target and nontarget schools). Th.e projects were 
designed to combat disproportionate disciplinary actions against minority 
students, identify some of the attributes of the more successful projects, and 
examine the reliability and validity of disciplinary data collected at the local 
level. 
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The study found that: 

o Preservice or related inservice training on the project was provided 
for staff in ^0 percent of the study sites; 

. o ♦ All of the ESAA projects provided direct services such as individual 
counseling and tutoring to students, and offered support services such 
as consultation and a home-school liaison to teachers and parents; 

o Three o^ the 15 projects showed a reduction in disproportion for 
suspensioh,>,expulsion, and corporal punishment in target schools; 

o The most ^ successful ESAA projects, operated under a central 
administrative structure, stated project objectives clearly and 
precisefjy, and used a planning process that included a needs 
assessment and participation among administrative staff, regular 
/instructional staff, and parents; 

'/ . 

/ - • r . M 

^ Measures of disciplinary actions were required by the Office for Civil 
Rights (OCR) and were the only measures , of disciplinary actions 
reported by all districts . : 



th|s study was designed to be descriptive in nature and was, therefore, only the first 
step in the diagnosis of the problems and solutions of discrimination in school 
/discipline. 

♦ ESAA Human Relations Services Improve Student Intergroup Relations, Attitudes, 
Behavior, and Self-Concept of Minority Students ^ ^ 

A recent study of ESAA-funded human relations programs reported that services 
provided by such programs in the schools were related to improvement in intergroup 
relations, behavior, and attitudes./ The study also reported that: ' 

o^. Schools with/ ESAA , human relations services showed greater 
improvement in intergroup attitudes, intergroup behavior, and self- 
concept of minority students than did schools without such services; 

0 Schools providing services directly to students showed more positive 
improvements in these ^reas than did schools where ESAA services 
were provided to only staff or parents; 

o Contextual factors were particularly important to the effectiveness 
of the services provided. In particular, district and school 
commitment to human relations was found to influence students' 
multicultural knowledge and attitudes; 

o Equal Educational Opportunity (EEO) was an important aspect of 
Classroom practices when teachers made conscious efiorts to further 
EEO i;^ their behavior , taward students and in their grouping 
practices. , 
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Based upon an'intenslve study of 12 sites to determine just how effective practices 
were made operational, a handbook will be prepared to provide guidance for schoofI5> 
interested in implementing human relations services. / 

♦ Lasting Change Has Been Brought About in Hundreds of Classrooms Across the 
Country " 

Preliminary reisults from a, recent Study of Dissemination Efforts Supporting School 
Improveme nt indicate that some common notions about schools and innovations can be 
challenged. Sixty-three innovations, for the^most part associated with Federally 
funded, improvement efforts, were studied through visits to 1^7 schools across the 
country. Teachers, administrators, and others involved in the implementajjion of these 
innovations, were interviewed to learn more about the realities and outcomes of school 
improvement efforts. Assessing the difference between present and past instructional 
practice, the study found that a substantial amount of change has occurred. Moreover, 
the innovations implemented by schools were not only faithful "replications" of the 
developer's idea, but they were remarkably stable in operating for more than 2 years 
without Federal support. ^ - 

A seriefs of reports describing the study's results and 'implications will be available in 
1982. 



♦ A Study of Teacher-training Programs in Bilingual Education 

The study findings are relevant to the goals of supporting bilingual education 
teacher-training programs in institutions of higher educatipn. Briefly, these 
§^oals are to produce a sufficient number of well*qualified graduates and to aid in 
the jbstitutionalization of programs so that they will continue in the absence of 
Federal funding. ^ 

Increasing the amount of funds available for stipends and fellowships would 
result in higher levels of enrollment at most institutions of higher education. If 
student stipends were terminated, the result would be a significant decrease in 
enrollment in many institutions. Since more students in Title Vll-aided^ schools 
than in non-Title VII programs are from minority groups, the former group would 
be more vulnerable to loss of financial aid. 

Graduates of bilingual education programs in States with bilingual education 
certification requirements not only meet State qualification standards, but would 
also meet and exceed the d^inition of qualified bilingual personnel as contained 
in the Title VII regulations.^ However, findings regarding required bilingual 
education course content and competencies suggest that program improvements 
could be made to enhance teacher qualifications. 

The amount and nature of coordination among academic departments has a 
significant influence on the curriculum. When bilingual education program 
personnel increased the curricular involvement of relevant faculty, the 
curficulum tended to be broader and more closely tailored to the needs of 
bilingual education students. 

Programs with a broad curriculum offered better preparation than those which 
emphasized a single curricular area. The quality and breadth of student 
preparation is likely to suffer in a program in which all the required courses are 
taught by one or two people. This probleipn is worsened in an institution of higher 
education with bilingual education programs at different academic levels, where 
the same few resources must be thinly distributed. Students at one level receive 
preparation that is highly similar to that of students at the other academic 
level. 

Higher education programs that address more than one target language, while 
meriting the goal of producing graduates of different ethnolinguistic groups, 
often have difficulty in achieving the same level of preparation for all groups 
represented in the program. Unless there are sufficient numbers of students in 
each ethnolinguistic group to justify additional faculty and separate courses 
geared to each group, program resources tend to be stretched too far to satisfy 
the academic and linguistic needs of all students. Institutions of higher 
education should be encouraged to collaborate to produce maximum effect from 
scarce resources. — 

The degree to which a program is able to sustain itself within an institution of 
^higher education is very largely determined by the number of enrollees. If ther^ 
are enough students to create a demand for the program, the institution of 
higher education will generally find money to support program faculty. 
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Other factors associated with institution alization are: 
o aictive support from administration; 

o positive attitudes on the part of non-bilingual education faculty; 

o some portion of bilingual education faculty supported by institutional 
funds; ' 

o some portion of bilingual education faculty on tenure track; 

o involvement of adequate numbers of professionals in .^program 
operation; i-' 

o compatibility of programs with institutional priorities; 

o a sufficient number of students to sustain the program without Title 
Vn support. 
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Pbstsecondary Education 



♦ Analysis of the Institutional Administration of Student Financial Aid Programs 
Using Data Collected in the Institutional Mail Survey 

This report^ the last in a series^ was commissioned to study the effectiveness and* 
efficiency of administrative practices in the delivery of Federal student 
financial aid programs at the institutional level. 

The study revealed that, on the average, ^-year public institutions have more 
recipients and employ larger staffs to serve student- aid recipients than do other 
schools. However, financial aid officers at ^-year schools also have heavier work 
loads than aid officers at other institutions, are more likely to be full-time 
employees, and are better compensated. The lack of sufficient compensation 
was found to be a serious hindrance to both the hiring and retaining of aid 
officers. 

Dissemination of student-aid information depended in large- part on the 
. availability of resources. It is an activity that needs considerable improvement. 

While there are many needs-analysis systems, the three that predominate are 
those of the Basic Grant Program, the College Scholarship Service, and the 
American College Testing Service. Validation procedures were found to be 
increasing, especially with regard to the Basic Grant Program. Information on 
income and dependency predominates. 

Finally, although practices differ on the packaging of student assistance, most 
are generally consistent with the principle of equity. 

^ This and other studies clearly point to the need for a detailed review, and 
perhaps a restructuring, of the Federal system of delivering student aid. As a 
result, several task groups have been at work on the issue in the Department. 

^ ■ ■ ' 

♦ A Report on Specific Federally Funded Graduate Education Programs 

This report on graduate education, which was mandated by the Congress, 
summarizes the operations of the Public Service Program, the Domestic Mining 
and Mineral and Mineral Fuel Conservation Programs, the Legal Training for the 
Disadvantaged Prograip, and the Graduate and Professional Opportunities 
Program. This rCBPrt also includes data on the special programs provided for 
female and minori^ graduate students, obtained from a spring 1978 study 
conducted for the then-Office of Education by the American Council on 
Education. The major findings of these studies are outlined below: 

o Graduate Programs Under Title IX of the Higher Education Act. 
These programs appear to be achieving their objectives of (1) 
increasing the supply of qualified public servants, particularly at the 
State and local levels, (2) supporting qualified disadvantaged students 
in obtaining advances degrees in domestic mining, and (3) increasing 
, the number of lawyers from disadvantaged backgrounds. 
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o Public Service Fellowship Program/Domestic Mining and Mineral and 
^ Mineral Fuel Conservation Program, Fragmentary dat^ suggest that 
the majority of former Fellows are employed in the^ respective 
' fields. \ 

o Legal Training for the Disadvantaged Program . Participants are 
graduating from law school and passing the bar on the first attempt 
at a rate which compares favorably^ to national norms. 

■ ' \ 

o Special Programs for Minorities and Females. A isurvey of 

postsecondary institutions which award post-bacculaureate degrees 
indicates that *6 percent had at least one formal prbgram 
specificailly designed for female and minority graduates in the spring 
of 197S. Programs included special admissions, student financial aid, 
and academic assistance. \ 

This report is the last in the series. Its requirement was removed by the^ Higher 
Education Amendments of 1980. / 

♦ A Study of the Dev loping Institutions Program (Phase I) ' 

The purpose of this study is to determine whether the program under Title III of 
the Higher Education Act can be effective in assisting institutions to provide 
programs consistent with student needs in anr effective arid efficient manner. 
Phase I of the study is intended to clarify program objectives and operations, c»nd 
to develop a study design for Phase II that will provide information for planning, 

budgeting, and program purposes. 

- ■ ■ i 

Preliminary research^cfocused on current program activities. It should be pointed 
out, however, that new regulations have been under development in response to 
the Higher Education Amendrhents of 1980. Although a new part in the law will 
4II0W for matching grants for endowment building, tHe program continues to 
provide funds with a high degree of flexibility for institutional development. 
While the flexibility makes it difficult to generalize about activities across 
institutions, it reinforces the concept that each institution must work within the 
confines of its own development plan. Thus, any generalization as to the overall 
effect of the program must be on the degree of success any institution has 
reached in moving toward "mainstream" status. 

Phase II will provide a series of case studies so that individual institutional 
responses to problems can be documented. Exemplary activities will be 
described and made available to other institutions. Successful implementation 
strategies will also be identified and made available to program staff for use in 
selecting projects and monitoring them. |jt , 

♦ Cooperative Institutional Research Program | 

Each year, more, than 300,000 college freshmen are surveyed under this 
cooperative program. It is funded in part by participating institutions and by the 
American Council on Education. The Department of Education sponsors the 
development of a data tape for analysis purposes and for program impact studies. 
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Last year, detailed tables were prepared for the Annual Evaluation RepQrt to 
show the difference in the distribution of Federal student aid, by program, 
before and after enactment of the Middle Income Student Assistance Act. ^^^^ 

Trends from another survey indicating that students from higher income 
families were benefiting from the program were validated for first-time 
students. In addition, the data provided information for a detailed technical 
paper describing first -time students by sex, race, family income, cost of 
education, and type of institution, 

♦ Evaluation of the Foreign Language and Area Studies Program (Phase I, 
Management Evaluation) y 

When the Congress enacted the National Defense Education Act (NDEA) of 1958, 
its objective was clearly articulated: *'to insure trained manpower of sufficient 
quality and quantity to meet the national defense needs of the United States." 
However, as a broader constituency made new demands on the program, its 
objectives expanded. It now includes not only specialist training but also 
international educational enrichment at all levels. 

The purpose of the phase I report is to analyze and identify program management 
and selected activities that maximize program efficiency* 

Findings by subprogram include these needs and recommendations: 

Centers . 

o Define potential grant recipients more flexibly; 

o Improve Center linkages with professional schools; 

o Define Center outreach requirements more flexibly; 



o Require that staff site visits be more thorough, that panel review 
criteria be clarified, and that attempts to recruit senior scholars for 
review panels be intensified. 

Fellowship Program 

o Determine employment trends of fellowship ' recipients and evaluate 
the match between training and employment; 

o Establish a protected competition for advanced students in the 
professions and high-demand disciplines; 

o Establish mid-career sabbatical awards to maintain and improve 
skills. ^ \ 

s 
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. , Graduate and Undergraduates Studies Program ^ 

It was found that forty -two percent of these seed-money projects were continued 
by the sponsoring institution after Federal funds, had ended. This is a much 
higher ' rate than for other Federal programs with a seed-money strategy. 
Successful seed-money projects should be identified and placed in the National 
Diffusion Network or other dissem^ination process. 

The second phase of this study, to be completed in ttie Summer of 1982, will 
analyze supply and demand trends for graduates in foreign language and area 
studies and assess the relationships between employment and program-supported 
training. 

Evaluation of the Special Services for. Disadvantaged Students Program 

The^iurpose of this study is to evaluate the impact of the Federally funded 
SpecialsServices for Disadvantaged Students (SSDS) program asoit existed during 
the 1979V80 academic year in postsecondary educational institutions across the 
country. ^This report summarizes the SSDS program's short-term impact on 
freshman^students who received special services from the program in that year. 
A followup survey will attempt to determine longerterm program impact on the 
same sample ^f students, many of whom will then be in their junior year in 
colleges and universities. Findings to dat'e include: 

o Students who received the full range of SSDS services were 2)i times 
more likely to complete the freshman year than similar students who 
did not receive such services; 

o SSDS students attempted and completed more course units than did 
the students who did not receive these services; 

o Full-service participation by SSDS students was associated with 
relatively lower grade- point averages in the first year. However, it 
is likely that this finding simply reflects thej fact that students with 
weaker educational backgrounds and weaker entry skills tend to be 
given more services. The fall 1981 followup study, when completed, 
should demonstrate whether or not this supposition is correct. 

♦ Loan Indebwdness Study 

The purpose of thikstudy was to develop a model to determine the ratio of total 
student loan burden\to discretionary earnings for program planning purposes. 
The primary components of the model are: 1) total cumulative loans expressed 
as a monthly repayment obligation; 2) estimates of first -year earnings and 
earnings for 9 successive years; 3) a measure of annual consumption i^xpenditures 
for each year of debt repayment, adjusted for increases in earnings; a measure 
of residual, post-consumption, discretionary income from which educational debt 
repayments can be made; 5) a ratio of annual debt repayments to available 
annual discretionary income. 

On ibe^ basis of historical :d|ita- that do not reflect recent sharp increases in loan 
amounts, the general cdnclusion was that college graduates, if erhployed full- 
time, would have discretionary income more thari adequate to repay their loans 



while maintaining an adequate standard of living. However, even with the 
earlier data, significant pockets of high loan burden were found among: 

o All baccalaureates who work less than full-time. They will be hard 
pressed to cover their repayments during their first 2 or 3 years out 
of school. Obviously, unemployed borrowers would face the complete^ 
loan burden while they remain without a job; ^ ' 

0 Married, full-time employed women with bachelor's degreek.' They 
will be substantially burdened during the first year or two unless the 
spouse is also working and has no debt of his own; ' 

o Health technicians, single or married. They could not support 
themselves at a low standard of living for the first 3 years after 
graduation, even if they had no education^ deb% Less severely 
strapped would be married farmers, housekeepers, and other service 
workers, who would Have to spend over 35 percent of their first year's 
discretionary earnings to r^pay their undergraduate education debt, if 
they were the sole wage earners. 

For a typical full-time employed, married, white female, the median annual 
repayment represented 70.5 percent of net discretionary income during the first 
year pf repayment but only 3.* perceint of such income during the 10th year of 
repayment. For full-time employed, single, white baccalaureates, the median 
firstryear repayment ratio was 13*7 percent and, in the 10th year, 1.6 percent; 
For black males, employed full-time, the comparable ratios were 6.9 percent 
(first-year) and 1.3 percent during the 10th year. All ratios assumed a low 
standard of living a$ defined by the Bureau of Labor Statistics (BLS), which is 
believed to be appropriate for newly employed baccalaureates. If one were to 
assume a moderate BLS standard, loan burden ratios would increase by an 
average of 99 percent during the firs^: year and by 23.1 percent during the 10th 
year of repayment. 

The study provides base line data for setting loan limits for Federal loan 
programs. * 

♦ National Longitudinal Study of 1980— Parent Survey 

The parent survey portion ofNhe National Longitudinal* Study of 1980 was 
sponsored. by the Office of Program Evaluation to determine the degree of 
commitment parents have in assisting their children to . finance their 
postsecondary education. Earlier studies indicated, after adjusting for family 
income, a high degree of variation in the willingness of parent? to contr*ibute 
toward their children's postsecondary education. 

The initial product from this study is a research data tape which, when matched 
with the student survey, will provide detailed answers to a number of related 
questions on the subject. The tape has been delivered and is available for 
analysis. An technical paper is under development by staff. 

♦ Sources of Loans Study ' 

The purpose of the study was to identify non-Federal sources of loan funds which 
parents and students could use to finance their educatiopal costs. The findings 
indicate that because of favorable terms to borrowers, the Guaranteed Student 



Loan (GSL) Program has become the predominant student lending vehicle over 
the past 5 years. Although there are several large, private, parent-loan plans, 

_the rates which these plans must charge are so high-xelative to^guar^nteed 

student loans that tLey are often used by families with one or morei children in 
high-cost colleges. 

The largest of these parent-loan plans had an increase in total dollar volume 
from $15.7 million in 1972 to only $33 million in 1979, with an actual reduction, 
ift the number of loans from 9,67* to 7,500, This was during the period when the 
GSL program volume ^vas increasing from $1.2 billion to $2.98 billion. 

Life insurance policy loans appear to be a fairly large source of family borrowing 
for postsecondary education. However, there has been onl^ moderate growth in " 
the volume of such loans, from $136 million in 1973 to an estimated $192 million 
in 1978. 

'\ 

Two huge, imtapped sources of capital fdr postsecondary education borrowing are 
life insurance company investment portfolios and corporate/union pension funds; 
These sources have not participated primarily because the student loan 
promissory hdte is not an appropriate funancial instrument, because individual 
loans are too small in amount, because the liquidity of a secondary market is 
restricted, and because there are too many diverse loan -servicing arrangements. 

♦ Field Test o; the Institutional Report Form (IRF) for State Regulatory 
Agencies 

The purpose of the field test was to provide State postsecondary education 
licensing agencies with a tool: to monitor the potential for student consumer 
abuse In the institutions over which they exercise authority. The field test was 
intended as a Federal technical assistance effort in response to one of ;the major 
recommendations of the 1978 national conference on State oversight. State 
licensing is the first step to institutional eligibility for Federal funds, and as such 
represents the primary means to protecting student consumers and the taxpayer. 

Response to the field test and IRF were generally positive. Fifteen of 18 
agencies contaeted agreed to participate in tHe field test, and all but 3 
completed their efforts on time. Sixty-eight percent of State agency and 
institution respondents had clearly positive reactions to the IRF and only 3 
percent had clearly negativ^ ones. Fifty percent of the respondents felt the IRF 
should be used more wide^yCand 21 percent opposed tms^-- ct\- 

IRF scores among the 77 institutions included in the field test were, on average, 
slightly better than in past field tests. Fe>yer problems were identified, and a 
higher level of awareness of the need for consumer protection seemed (Bvicj^nt. 

The IRF was most useful at new schools, nonaccredited schools, and others that 
do not already undergo a regular review. The schools preferred it as a self -check 
educational tool for institutional officials, rather than as a monitoring device to • 
be used in an authoritorian manner by the State agency. 

The IRF is being used by IState licensing agencies, private accrediting 
associations, and by postsecondary education institutions as a means of gauging 
their policies and practices in student consumer protection. 



♦ Trends in Financial Indicators of ColJ eges and Universities . 

„This~H^ort"presents the--resul%s-af a sur-vey-at -the - financial -condition af a 
nationally representative group of colleges and universities using two sets of 
financial indicators for a 5-year per^pd (197*-5 to 1978-9). One set measures 
changes in the institutions' financiall assets while the second is concerned with 
changes in human resources, i.e., faculty, students and administrators. 

Highlights of the findings include: 

o Private *-year colleges show a decline in their ability to meet 
current fund liabilities with current fund assets; 

o The financial reserves of private *-year colleges and universities 
have fallen more than 10 percent over the' 5-year study period, 
suggesting less long-term stability; 1 

o Four-year colleges and universities have had an increasing pool of 
potential students to select from, while the pool for j2-year colleges 
appears to be declining; 

o Dormitory occupancy rates have remained fairly^ stable or have 
increased slightly over the past 5 years, minimizing the possibility of 
' vacant dormitory space being a financial drain; 

o The proportion of expenditures used for salaries in public institutions 
has increased over the 5-year period while it has fallen in private 
institutions, suggesting less budget flexibility in the public sector; 

0 Not-for-credh enrollments increased substantially in both public and 
private universities, suggesting that greater outreach activities are 
being employed; 

o Institutions apparently have been able to maintain the number of full- 
time-equivalent faculty, suggesting no apparent reduction in course 
of ferings available tb students. 

The findings, while suggesting thatvbelt tightening continues, also indicate that 
most institutions are becoming more efficient in ^he use of thdr resources. 

♦ The Financial Conditions of Institutions of Higher Education 

In 1973, a number of studies estimated that as many, as 71 percent of collegiate 
institutions were in or beaded for financial trouble. Now, nearly 10 years later, 
the vast majority of this 71 percent survives. However, what of the future? A 
study was conducted to identify irv detail the, reasons for the present financial 
condition of colleges and universities. The study found: 

o Reserves are down for private colleges; 

o Flexibility is disappearing for the public institutions; 
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o Scientific equipment is agings 
o Endowments can no longer provide the subsidies of the past; 



b Administration takes a larger curid la^ amount of the education and ^ 
general budget; 1 ' 

o Capital assets are increasingly in tieed^of revival. 

It was also indicated that Federal student aid has bren the best vehicle for 
Federal aid to higher education since it provides financial support with a limited 
degree of Federal control. ^ 

The study team also found that problems with institutional finances have slowed 
but not stopped progress toward national higher education goals. Progress has 
been made toward all goals, from access to institutional excellence. However, 
there are doubts about th^ ability of institutions to maintain this progress, given 
continued financial difficulty. In particular, the study suggests that declines in 
Federal student financial aid could trigger declines in enrollments which could 
have a very profound effect on a number of institutions. In this respect, the 
study report calls for the States to do a more comprehensive job of monitoring 
the health of both public and private institutions. 




'61 



6i 



Special Category Education 



♦ The State- Administered Program of the Adult Education Act 

A small scale assessment of the State-Administered Program of the Adult 
Education Act was completed in 1980. While there are deferences in estimates 
of the size of the adult target population, there is evidence that the programs 
designed to serve this' population have been effective. During the 1978-79 
program year, almost 2 million participants were served the program at an 
average cost of $*6 per participant, with over 60 percent of them under 35 years 
of age. Almost half of the participants were employed an J 27 percent received 
some form of public assistance. Some 53 percent of the participants had nine or 
. less years of prior formal schooling* 

One of the key issues that are of primary concern to/ State and local adult 
educators is the clarity of the program's purpose to /educate disadvantaged 
adults. The areas emphasized by State directors of adulr education have directly 
reflected Congressional intent as expressed in the 1978 Amendments to the 
Adult Education Act. 

Over 90 percent of ,the State ^directors of adult educat/on surveyed indicated that 
literacy and consumer competence are areas their/ States have emphasized. 
Some 86 percent reported they have placed a priority on developing fum^tional 
pre-employment 'skills as well. In Addition, 75 percfent of teachers and project 
directors indicated that competency -based instr/iction is important in the 
education of adults. 

♦ The Regional Resources Center Program / 

A study of the Regional ResouVces Center (ReIc) Program to assist State and 
local education agencies in implementing P.L/. 9*-l*2 was completed in June 
1980. Over 31,000 administrators, teacher*, and parents received training 
services from the RRC Program in I978-79J with emphasis on Individualized 
Educational Program (lEP) development. RRC's were providers of service in 30 
' percent of the school districts surveyed while State agencies were providers of 
service in *9 percent of the districts. The types of RRC services that have had 
the/ widest appeal and utility have been training and other assistance directed 
towards capacity building or training of trainers at the State agencies. The 
majopffcy of State agencies surveyed indicated that as a result of RRC services, 
Vthere was added capacity in the agencies to provide educational materials, train 
teacners, and improve the implementation of P.L. 9*- 1*2. The multi-State 
»RC's seem to be more successful and more effective than the single State 
RRC's in building State agency capacity to implement P.L. 9*- 1*2. 



Teacner Corps 



The current Teacher Corps evaluation was designed to coincide ^^ith the 5-year 
Jijnding strategy of thg^program. The developmental charactep'^ of the program 
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suggested an evaluation that would utilize a variety of methodological 
techniques, one of which was ethnographic. The first major product from the 
longitudinal study was an assessment of collaboration and multicultural 
education processes and practices developed in Teacher Corps. Collaboration 
and multicultural education were specific legislative concerns that were to be 
addressed within the projects. The study revealed that the concept of collabora- 
tion was materializing in the projects at all levels' — i.e., community, schools, 
universities, and school districts. 

The concept of multicultMral education was difficult to articulate and implement 
at the project level. Teacher Corps interns and team leaders could be 
effectively used at this level to develop specifics of multicultural education 
programs, because pf specialized training they receive upon entering the Corps. 
The most serious impediment to the develo ment of multicultural education 
programs was the participants' misunderstanding of the goals and objectives of 
multicultural education. 

♦ Community Education Program 

This study, completed in February 1981, examined the level of develop ment "of 
States' community education programs since the Enactment of the Community 
Schools Act of 197*. The study found that: 

o The strongest components of. community education programs at the 
State level were State needs assessment and interagency cooperation; 

", (■, 

o The weakest components were in $tate planning, evaluating, and 
reporting; ; 

o Only about half the States have strong commitments to community 
education through supporting legislation or State funds;. 

'f 

o State education agencies with full-time community education 
CQordin^tors are more likely to have Sta^e-level operations in place; 

' ■ . . \ 
o At least 2 consecutive years of Federal funding appear to be the most 

effective cost/benefit pattern for State education agencies; 

' ' . • . . \ . ' 

o The need exists for data to be collected on local community 

education programs and then to be aggregated a^ the State and 

national levels. 



♦ Special Em|>hasis Projects of the Right-To-Read Program 

This recently completed , study was based on observations at seven matched- 
school sites, on reaping remediation through a prescribed pattern of instruction. 
The study found /that of the three sites which compjied closely with the 
prescribed pattern, two showed significant reading achievement gains among 
second and fifth jgraders. Reading achievement was measured among the segond 
through the sixth grades with the Stanford Diagnostic Reading Test; only the 
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second and fifth grades revealed consistent gains at the two sites. The major 
features of the prescribed pattern of instruction /^ere the use of reading 
specialists for all children in, the first and second/grades, the use of reading 
specialists for all children with reading problems in/grades three through six, and 
institution of a vacation-time reading remediat^pn program for children who 
were reading below grade level* 

■ ■ . / / 

♦ The Inexpensive Book Distribution Program Off the Right-To-Read Program 

The Inexpensive Book Distribution Program (ffiDP) was begun through a contract 
to Reading is Fundamental (RIF), Inc* The^^IF objective is to provide books^ 
children which are of maximum interest to them, then allowthg^jsinld!^^ 
choose their own books from a large numbejp of titles^^JDoe^assOrnption is that the 
children will be more motivated to read bec^iise-thSy' setected^theJ^ooks^in which 
they were interested. In fiscal year 1979, RIF provided . support to; 1,8^2 
subcontractors who, in turn, were responsible for purchasing the books and 
organizing distribution of them on a schoolrwide basis. 

/ 

In 1978, the Office of Program Evaluation initiated a study of the IBDP which 
was completed in 1980. The objectives of the study were to determine the 
effectiveness of the IBDP in reading motivation, ''and to describe the process by 
which books are acquired and distributed to children. 

The study could not be completed as originally designed due to unexpected costs; 
thus, the first objective^ was not fully accomplished," 

The findings of the study were as follows: 

o The program is enthusiastically endorsed by school personnel, 
parents, volunteers, and sponsoring community agencies and or gani- 
' zations, at all the program sites visited in thi^ project; 

o Strengths of the program were perceived to be book ownership by the 
children, enthusiasm for reading among the children, community 
involvement, and increased silent reading by the children; 

o The problems fell in two categories. Some were related to the books 
(e.g., storage, ordering, payment procedures, and increasing prices of 
books). Others were rela;ted to the program (e.g., subcontractors 
desired more technical assistance from RIF, bookkeeping was 
' becoming more cumbersome, fund raising was more difficult, and 
there was a negative attitude toward Federal programs). 



♦ Emergency School Aid Act's pZ-Program 




An evaluation entitled "Assesfement of the ESAA-TV Program, through An 
Examination of Its Production, Distribution, and Financing", found that the most 
serious challenges for this program are obtaining increased viewer ship at home 
and obtaining increased utilization in schools. j 

The report stated that ESAA-TV has filled a gap by funding the production of 
television series which meet the needs of minority audiences while providing a 
valuable alternative for majority viewers. 
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It recommended that the' program be continued, and that the level of funding for 
production and ancillary activities be increased to ensure that quality 
programming will be produced and viewed. 
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♦ Title I of the Library Services and Construction Act 

The purpose of this study was to evaluate the impact and effectiveness of Title I 
of the Library Services and Construction Act (LSCA Title I), which is the only 
federal program designed to assist State efforts to develop and improve public 
libraries. Findings indicate that: 

^ o The most significant area of change induced by L SCA T itle I has been 
the establishment of regional systems of pub+itTlibraries, &i some' 
instances, these expenditures have been translated into direct 
services to the piiblic through bookmobiles or books-by-maiL 
However, the major expenditures are aimed to improve the ability of 
regional libraries to assist local libraries through interlibrary loans 
and delivery systems and to install cost-cutting practices such as 
centralized purchasing and cataloging, 

A major impact has been the provision of services in rural States and 
high poverty States, '\ 

The program has had a significant effect on the establishment and 
extension of public library services for residents of State-supported 
institutions, the blind and physically handicapped persons. 

Adequacy of public library service represents current and dominate 
focus of most State efforts to improve public libraries. Complicating 
all these efforts, and especially the upgrading of existing services, is 
cost. Increasingly, the ability to pay is being outstripped by cost, 
causing a shift from the goaTcJf improvement toward maintaining 
existing levels of services. 

While only 33.9 percent of all public libraries in the nation received 
at least one direct LSCA Title I grant since 1965, it is estimated that 
only 6 percertt of all public libraries failed to receive at least one 
direct benefit from the prograrh. Public libraries receiving at least 
one grant.served an estimated 69 percent of the nation's population. 



Vocational Education Programs for Indian Tribes and Organizations 

The Vocational Education Act of 1963, as amended by th^ Education 
Amendments of 1976 (Public Law 9*-*82), provides a one-percent set-aside for 
vocational education programs for Indian tribes and organizations. The program 
is discretionary. 



65 



/ 



This study was mandated by the Technical Amendments of 1977 (Public Law 95- 
*0). It focuses on the first two years of the operation of the program. It Nyas 
designed primarily to examine how the program was working in its formative 
years and to provide information needed to improve performance. Data 
collected in the spring of 1980 describes 17 projects, needs assessments, progrd^m 
operation and performance in their second year of operation. \ 



The study shower* that: 

o Projects^ were primarily located on or near Indian reservations in 
communities with Indian labor force unemployment ranging from 26% 
to 50% to more than 75%. These communities had a high percentage 
of Indian adtflts who hid not completed the 8th grade. In a majority 
of communities, the program provided services and activities in 
Indian communities where no such services and activities exfsted 
previously. 

, i o A high correlation existed between the^ local identified training needs 
V and the project training components designed to meet those needs. 

^ However, the less sophisticated tribes required assistance in linking 

vocational training needs to the economic development of the tribes. 

o AKhough study follow-up information was not available, trend data 
indicated that 39% completed their training and secured jobs or 
continued their education during the program year 1978-79, and ^8% 
completed their training and secured jobs or continued their 
education in program year 1979-80. Program reports indicated that 
projects have continued to improve their performance. 
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CHAPTER IV 



Evaluation Activities in the Officie of 
Management 



ERIC 
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Organizational Perforniance Services 



In this period of a shrinking national budget and declining confidence in public 
education, the Department of Education is acutely aware of the need to increase 
the performance^ or results of Federal programs, and to improve accountability 
mechanisms through which these results are measured and reported. To this end, 
|he Qffice of Management is designing and establishing a Department-wide 
Performance Accountability System (PAS) which will provide a common 
framework for defining objectives, measuring outcomes, and using information to 
strengthen Department management. The PAS will include the support functions 
oi the Department such as personnel, fiscal services, contracting, and planning 
because effective operation of these functions is critical to achievement of 
program objectives. 

The Office of Organizational Performance Services (OPS), located in the Office 
of iManagement, is responsible for installing and managing this new sys tem. The 
Division of Program Assessment in the Office of Per for mance^3er vices, is 
ch)arged with the special task of assisting ED'S program managers to develop 
appropriate objectives ''and indicators for the new Performance Accountability 
System. OPS also designs and conducts rapid turn-around decision-oriented 
studies which provide managers with timely information about program 
performance so that early corrections can be made. To carry out its assignment, 
OPS employs several specific techniques: Evaluability Assessment, Program 
Performance Design, Service Delivery Assessment, Program Management 
Review, and Rapid-Feedback Evaluation. 

These techniques are not evaluations in the traditional sense and do not 
substitute for necessary impact evaluations. However, they can help identify the 
effects of programs at State and local levels, and pinpoint areas where ED 
managers and policymakers can influence program outcomes by changing certain 
aspects of the Federal involvement. The Evaluability Assessment and Prcgram 
Performance Design techniques alsb help lay the groundwork for planning and 
conducting more traditional evaluation studies at appropriate points in the life of 
the program. 



Evaluability Assessment is designed to improve program outcomes and the 
evaluation process by first ensuring that a program has a solid management 
foundation. An EA determines what changes might be needed to make the 
program more manageable and accountable; the extent tp which a program is 
ready for ev^jation; and how an evaluation of the program might be most ' 
usefully conducted. A fully successful evaluability assessment results in: (1) 
clearly -^specif ied and agreed upon program objectives and activities; (2) an 
explicit statement of the assumptions that underlie the program; (3) a list of 
program performance indicators or measvires that are agreed upon by those 
responsible for the program; and (*) a plan with both management and 
measurement options which may be implemented as program managers see fit. 




An EA starts by considering questions such as the following: 




What are management's program objectives and expectations? What 
resourced, activities, objectives, and assumptions make up 
management's intended program? 
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o What, in the view of policymakers (executive and legislative), is the 
program expected to accomplish and what are acceptable indicators 
of performance? 

o What are the likely uses of information on program performance at 
each manai^ement level? What range of actions might management 
consider as a result of various findings? 

On the basis of responses to these questions and a review of relevant documents 
(e.g., the authorizing legislation wd legislative history), the assessment team 
develops three types of descriptive charts or models of the program. A logic 
model is drafted to represent the intended logic of the program (e.g., if event A 
occurs, then it is assumed that event B will occur). A second 3et of models, 
known as function models, trace the program's processes, including such events 
as^ flows of activities, people, money, and information. The third model deals 
with measurement of progress toward program objectives. It identifies measures 
which could be taken at various points in the- pif-ocess to indicate program 
performance. Since, all these models are based on interviews with Federal 
-^f f i i Is only, they represent a description cfi[ the 4ntended program . 

The assessment team then visits a limited sample of projects to obtain 
information about the' actual program as it operates in the field. A second set of 
models is developed to represent program reality. Questions addressed during 
and after the f.eld visits include: 

o What are the program inputs, activities, and outcomes based on a 
review of actual operations at selected project sites? 

o What measurements and comparisons are feasible, given existing data 
systems? 

o ' What data are obtainable on program performance? Are there data 
sources for management's qigreed-upon measures of progress? 

All information gathering and modeling at the policy, nrianagement, and field 
operations levels lay the groundwork for the analytical stage of the assessment. 
Questions asked by the team during the analysis of program evaluability include: 

o Is management's description of the program well-defined, acceptable 
to policymakers, and is it a sound representation of the program as it 
exists in the field? 

o Are management's expectations plausible — that is, do they appear 
likely to be accomplished? 

o What portion of the intended program is ready for useful impact 
evaluation? 

o What management options can be suggested to improve program 
per for finance? / 

* \ / . - 

Like full-scale impaict and process evaluations, EA's are conducted by an outside 
team of researchers^ Subject to close supervision by a Department project 
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monitor. Evaluability assessments differ significantly in that they are intended 
to produce results relatively quickly (6 months) and inexpensively (an average 
cost of $67,000). An added benefit is the savings of time and money that might 
otherwise be expended on unsuccessful process or impact evaluations, that is, the 
resources that might be used to evaluate a program shown by an EA to lack 
agreed-upon, measurable objectives. 

Twelve EA's were initiated in FY 1979 and FY 1980, and 6 more were initiated 
during FY l?81, making a total |f 18. Two of the early EA's weri terminated 
before completion (Vocational Ed cation, and the National J Center for 
Educational Statistics). Ten of the assessments had been completed as of the 
end of Jiine 1981, and 6 were still in process. The 10 completed studies covered 
the following programs: Bilingual Education, Career Education, Cooperative 
Education, Early Childhood Education for the Handicapped, Follow Through, 
Independent Living Centers, Institute for Museum Studies, Language Training and 
Area Studies, Vocational Rehabilitation, and Women's Educational Equity. At 
least three new assessments will be undertaken during FY 1982.-S (See Volume II, 
Appendix B for more information on EA's.) ~ \ 

Two Rapid-Feedback Evaluations, an optibnal second phase of EA's, were also 
initiated in FY 1981: one on Career Education, and the second on Women's 
Educational Equity. The first was completed in June 1981 and the second will 
end in November 1981^ Rapid-Feedback Evaluations use readily available data to 
follow up EA's by exam^ing furtht^r the evaluable parts of a program. 

Program Performance Design (PPD) 

The Program Performance Design technique, like Evaluability Assessment, aims 
to generate measurable program objectives and performance indicators, but is 
less intensive and less costly. This approach places primary responsibility on 
program managers for identifying key objectives, developing measures or 
indicators, and preparing required progress reports. OPS assists by defining 
terms and setting criteria, giving training and technical support, coordinating 
departmental review, and advising on the quality and appropriateness of final 
products. 

As part of the Performance Accountability System, PPD will help managers 
produce objectives and indicators just prior to ED's internal budgeting cycle so 
that the objectives are useful in the budget review and policy activities of the 
Department. In addition, the Department will be able to improve its response to 
the mandate of Section *17 of the General Education Provisions Act requiring 
development of specific objectives and performance indicators for all ED 
programs. / ^ U 

The objectives and indicators developed through this process will be transmitted 
to the Congress in the Annual Evaluation Report. It is anticipated that the 
^^^smission of these individual program objectives and measures will assist the 
Congress and the Department in agreeing upon legislative and administrative 
mandates for program operations. Some differences will naturally emerge in this 
dialogue, but the existence of the initial objectives should be helpful in clarifying 
and resolving those differences. 
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Programs which are especially complex or controversial will have objectives 
developed through the mor^ analytical, intensive process of Evaluability 
Assessment. Whether developed by the Evaluabilit^ Assessment or the Program 
Performance Design technique, program objectives an J indicators — as part of 
ED'S Performance Accountability System — will be closely tied to performance 
appraisal and merit pay, thus reinforcing the accountability of managers for 
program performance. i 

Program Management Review (PMR) 

This evaluative technique is also bein*' develooed and re^'ned bv ^h*» ivision of 
Program Assessment. These studies are designed to synthesize exist ' B program 
information in a short period of time and to supplement that information with 
independent, short-term assessments covering important knowledge gaps about 
the operations of the program. The resulting report gives the Secretary and 
other top managers a complete but succinct picture of a program including its 
history, legislative goals and objectives, resources, administrative and 
management operations, field perceptions, effectiveness, issues, and problems. 

Program management reviews will be particularly valuable when programs are 
facing reauthorization without sufficient data on certain aspects of their 
operations; when controversial issues require (^ficult management decisions; 
and when top management changes are made. While PMR's are not designed to 
develop quick solutions for program problems, the/ can be used as diagnostic 
tools to identify where policy, administrative, or legislative changes should be 
made. 

■ f 

A Program Management Review will be directed by OPS staff with the 
assistance of other program and staff offices within ED. Staff with expertise in 
management and program analysis, work measurement, organizational behavior, 
quality control techniques, and other fields make up a PMR team. 

The specific purpose, scope,, level of detail, and length of each PMR wUl be 
defined by OPS in conjunction with top management and program officials. 
Because each PMR will require a significant level of resources, no more than 
three or four such studies will be undertaken in a fiscal year. Use of the PMR 
technique will be reserved for high priority needs and interests of the Secretary, 
Under Secretary, and Assistant Secretaries. It .is expected that performance 
shortfalls highlighted through the Performance Accountability System would lead 
to a top management request for a Program Management Review. 

Service Delivery Assessment (SPA) 

These short-term, current assessments of ED orograms and program-related 
issues are to be conducted in fiscal year 1982 in relation to issues generated by 
the Performance Accountability System, or at the special request of the 
Secretary or Undersecretary. These 3- to 5-month studies document the 
experiei;ices and impressions of persons who directly receive or provide services 
under ED programs in order to gain an understanding of how xprogram 
performance is perceived in classrooms^ in State offices, in the local^ school 
boardroom, or at colleges and university campuses. An SDA gauges how 
successful ED programs are perceived to be at the State and local levels in 
rpeeting the needs of students, teachers, parents, and administrators; what 
fJroblems exist as programs are being implemented; and what possible 
improvements are identified by those directly affected by the Federal programs^ 
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It is important to understand that SDA's do not correspond to traditional impact 
or process evaluation studies, audits, compliance reviews, or monitoring 
activities. While they often employ similar methods, SDA's are more analogous 
to in-depth analytical reporting which uses open-ended discussions with people in 
local settings. The knowledge gathered is generally subjective and qualitative in 
nature. Because of these characteristics, SDA findings are not-injtended for 
general distribution but rather for use along with other program information 
sources available to the Secretary and other decision-makers. 

The reasons for undertaking an individual SDA vary with each study and may 
include continued gaps in the Performance Accountability System, suspected 
operational problems, significant changes planned or underway, expiring or 
proposed legislation, plans for a major initiative,^ or programs' or issues the 
Secretary wsints to explore. The SDA teams seek especially to identify 
operational improvements which the Secretary and program managers can make 
without the need for legislative, regulatory, or budget changes. In this fashion, 
the SDA findings and recommendations can be implemented immediately to 
improve program performance. ^ 

In the Education Department, SDA's will be conducted under the overall 
supervision of OPS with regional staff providing onsite support necessary to 
conduct the studies. It is hpped that State and local education staff will 
eventually participate with ED staff in planning and executing these studies. 

Because SDA is a new. activity relative to Evaluability Assessment, only two 
major stud^ies were begun in fiscal year 1981. The first was^ a joint national 
report by the Department of Health and Human Services and the Department of 
Education on services to refugees. The second was an examination of the 
National Direct Student Lban Program which was supervised by OPS staff and 
conducted by the personnel from four regional offices. It involved students, 
parents, representatives of loan companies, collection agencies, university staff, 
and lawyers. As new policy begins to change the Federal^ relationship to State 
and local education agencies, it is expected that SDA will meet a definite need 
of top policy-makers for direct, timely feedback on how these changes are 
affecting the educational enterprise throughout the country. 
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Education Data Control 



One of the main functions of the Office of Organizational Performance Services 
is education data control. This function involves a review of data collection 
activities and instruments of the Department and of other Federal ag^encies. The 
Federal Reports Act of 1942, as amended, the Control of Paperwork 
Amendments of 1978, the Privacy Act of 197*, the Paperwork Reduction Act of 
1980, five Office of Management and Budget (0MB) circulars, and Departmental 
regulations for contracts and grants, are the framework for the OPS role in 
forms clear^ce. These guiding documents define many of the requirements and 
responsibilities placed on the Department, on OMB, on the Federal Education 
Data Acquisition Council (FEDAC), and on all Federal agencies collecting 
education data from the public. / 

Before the Department of Education was formed, component offices ot the 
Office of Educlation (OE) submitted each gropbsed form for clearance to the OE 
Office of Management's Administrative ydmpliance Staf! (ACS) for review. 
That staff performed a/^reliminary review and worked with program form 
sponsors to correct and improve package^and to reduce burden hours imposed on 
the public by each form. The. forms clearance package was then subrfiitted by 
Administrative Compliance staff to FEDAC for approval as required by the 
Control of Paperwork Amendments of 1978. FEDAC staff then hegan anew to 
conduct their own review of the clearance package. 

In each review, many of the same steps had to be done by FEDAC that had 
already been done by Administrative Compliance Staff. xMs duplication of 
effort w^ recognized by departmental management in 1980, a^ it was decided 
that the two functions should be merged. Early in 198f; the Division of 
Education Data Control was formed with staff members of both the 
Administrative Compliance Staff and FEDAC. The unit was given responsibility 
for forms clearance as well as for the ED Information Collection Budget, 
interagency repdN:^ management, the Hatch Amendment, copyright approvals, 
environmental impact approvals, and the Family Educational Rights and Privacy 
Act. 

At about the same time, the Paperwork Reduction Act of 1980 expanded the 
scope of FEDAC control over data collection activities. Where FEDAC had 
previously only been involved if (1) the primary respondents were education 
agencies, and (2) the purpose of the data collection was related to education 
policy, research, or evaluation, the 1980 Act broadened the field by requiring 
FEDAC approval whenever either of these two conditions exi^. 

The Paperwork Reduction Act of 1980 also created other new requirements 
related to forms clearance. The most significant of these are the requirements 
that OMB approve every data-collection proposal, that 0MB announce ec.ch one 
in the Federal Register , and that OMB approve or disapprove any request for 
clearance within 60^ calendai* days. Because these requirements could have 
reinstated a double review (once by FEDAC and once by OMB), an agreement 
was reached with OMB so th&t, for the purpose of forms review; FEDAC will act 
as an arm of that agency. ' 

Coupled with the publication each February of proposed data collections required 
by FEDAC legislation, the ED Information Collection Budget is of use in 
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reviewing planned Federal data acquistions. Arrayed in various exhibits 
(including type of collection an4 respondent), accumulated burden hours provide 
a comprehensive outline of the departmentaf strategy for collecting information. 
Types of collections (such as applications for benefits) can be identified and ' 
targeted for reduction much more realisticallV and efficiently! when they are 
presented in the aggregate. 0MB and OPS are closely examining the 
relationships between dollar cost and burden hours. The capacity may ^oon exist 
to tie dajta collections into specific appropriations and operating expenses. 

/•' ' [ ■ ■■ - • 

Administration of the Family Educational Rights and Privacy Act of 197*/ 
(PfeRPA) is another \esponsibility of OPS. The purpose of the^ FERPA is td 
/stablish^ a national minimum standard for the recordkeeping practices of 
/educational agencies and institutions- which receive funds ^nder the 

/ administrative authority of the Secretary of Education. The la)v-^complished 

' this purpose by requiring schools to: 

6 Motif y parents and adult students of their-rights under the FERPA; 



o Adopt written records polipies consistent; with the minimum 
requirements set out in FERPA's regulations; \ 

o Permit parents and adult students (inducting former stfudents) to 
review and inspect education records; 

o Permit parents and adult, students (through form-il hearing 
pt'ocedures, if necessary) to seek correction of recorded information 
which they believe to be inaccurate or misleading; 

o Permit parents and adult students to exert control over distrlosures of 
information from education records through a consent requirement 
unless such disclosures are specifically permitted by the FERPA. 

The role of QPS is to provide information to tho§e affected by the law and to 
investigate complaints alleging violations of the law. On the average,- the office 
responds to about 5,000 inquiries a year. These come from parents, students, 
school administrators, attorneys, members^of the Congress, State departments of 
education, other Federal agencies and, in some instances, representatives of 
foreign governments interested in developing similar privacy laws. 
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Management* Analysis 



Primary efforts in management analysis tor date have been directed toward 
improvements in administrative and management systems which suppor . program 
operations. Although these systems indirectly af fept program operations, they 
nevertheless can have a significant impact on program performance. The 
methodology used in conducting studies is based upon thfi ideas and concepts 
formulated by W. Edwards Deming^ a pioneer in utilizing sitatistical methods to 
improve productivity and quality control. 

I' ' " ' 
In addition to productivity improvement studies and office technology research 
and development, the OPS performs ad h9Q research and analysis on a variety of 
management issues^ Of particular significance during the past year was a special 
report to the Secretary of Education on improving the delivery systems 
supporting the Student Financial Aid programs. This project was undertaken -as a 
joint effort with the Office of Program Evaluation, Division of Postsecondary 
Programs, and the Information Resources Management Division. 

The Federal student financial aid loan and grant programs distribute nearly $9 
billion in aid annually to over 6 million students. The delivery systems that 
manage and administer thes^ programs were not designed to handle programs of 
this magnitude and have / not kept pace with their rapid acceleration. 
Consequently, the systems today have serious deficiencies that jeopardize the 
effective delivery x>t studeijit aid and leave the programs vulnerable to many 
forms of fraud and abuse. 

The sp^ial report to the Secretary strongly recommended that he assign high 
priority to undertaking a structured system .development approach which would 
include establishing dedicated Project Office reporting directly to the Under- 
Secretary, and sei^^ting an Advisory Committee^comppsed of experts in the 
fields of education^ finance and systems design. 

Interdepartmental Office Technology Demonstration Pro jet t 

The purpose of the demonstration project is to study the product^jdty of Federal 
employees when given new technology. It is sponsored ^^^he Office of 
Personnel Management, the Office of Management and Budget, and the General 
Services Adminiistration. ifKjgencie^ participating in . this projec;t were the 
Department <j>f Education, the Department of Housing and Urban Development, 
of Commerce (the National Telecommunication and Information 
and the Office of Personnel. Management. ^ , / 



the Department 

Administratijdn), 



As a participant in this project, OPS is responsible fo^r developing a, detailed 
project plan ahcl phocedures'^ for managing the project; conducting a feasibility 
study and identifying productiyiry improvements, which includes documenting 
.current systems; and misasurlng, productivity, and simplifying work flow and 
systems .before automating; designing office automation systems with three or 
more office autom^' ioh technologies; and implementing automated systems, 
documenting results, and measuring effects including productivity changes. 

During this sti^dy ^ effort will b* made to identify , and examine the '"people 
factdrs" and their implications for (successful implementation of advanced office 
technology and procedures. These personnel management factors include job 



enhancement, job satisfaction, motivation, classification and position 
management, and psycho^gical impacts. OPS will also be analyzing technology 
capability factors Ithe capabilities, cost, and benefits of selected Information 
handling ' equipment and technologies), and institutional factors (identifying 
organizational staffing, budgeting, and management issues and evaluating the 
/impact resulting from new information technology). 
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Organizational Development 

Organizational developmenl 
management-by-objectives, 
Communications System), 
effectiveness^ 



activities within the Department include 
management directives (Administrative 
organizational analysis, and organizational 



^ Administrative Communications System (ACS) 

Enhancing the effectiveness and efficiencV of any organization depends to a 
* great extent on how well it develops and disseminates its administrative policies, 
procedures, and other important information. Additionally, 1 the Federal 
regulation "Establishment and Management of an Effective Directives System," 
mandates the creation of a management directives program in each Federal 
agency. The ACS is the official means of communicating administrative 
policies, procedures, and information affecting mc^re than qne principal office 
within the Department of Education. The ACS includes manuals on general 
. administration; mission and organization; grants, jprocurement, and financial 
management; and perspnnei and administrative management. ^ It also sets forth 
approval authpriti^^ for permanent Departmental directives, temporary 
Departmental directives, and Departmental announcements. 

The Office of Organizational i'*erformance Services has developed the 
Department's policy on organizational design. The Office is responsible for 
evaluating all proposed organizational changes to assure that three primary 
objectives outlined in the ED policy are met: (1) improved delivery of program 
services to the public, (2) improved internal operations, and (3) mor^ efficient 
use of staff/resources. * 

A major/activity close^ linked to organization design has been the dev€?lopment 
of an^Eb-rwide system for Delegations of Authority. Still in the final phase of 
development, the system, when automated, will provide information on the 
position location of various program and administrative authorities' within ED. 
This information is critical to internal operations in that signature Authority is 
necessary the awarding of grants and contracts and the execution of 
administrative documents. ' 
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Organizational Effectiveness 

The organizational! effectiveness activities assist individuals and ED units to 
work more effectively and productively. The unlt^does this by designing and 
conducting activitijes with individuals and groups aroijnd such themes, as 
diagnosing organizational problems, clarifying communications, establishing 
goals, working with conflict, improving meetings, solving problems, improving 
decisionmaking, and improving interpersonalrielations. - 

Specific activities include: i 

o Leadership transition assistance to aid new leaders to take command 
quickly and effectively. 

o Assistin^\ key staff of politically appointed leaders to master their 
roles and\to learn how to work efiectively with the bureaucracy. 
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Designing and conducting team-building activities for leaders, 
managers, and their staffs. ^ 

Conducting training in organiza\:ion development areas such as time 
management, conflict resolution, affirmative action, leadership 
styles, effective communications, and conducting productivie 
meetings. 
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